A method, electronic device, and chip system for recognizing indoor and outdoor scenes.

Abstract

This application provides a method, electronic device, and chip system for identifying indoor and outdoor scenes, relating to the field of coarse positioning technology. The method obtains the walking characteristics of a user carrying the electronic device by acquiring time-related data collected by multiple sensors on the electronic device. Then, based on the user's walking characteristics and the number of samples, it determines prediction modules related to the current walking characteristics. Through these prediction modules and data related to these prediction modules from the data collected by multiple sensors, it obtains corresponding prediction results. When multiple prediction results exist, they can be fused. After fusion, the fused prediction results can be corrected based on satellite data to obtain the identification results of indoor and outdoor scenes. This method can accurately and quickly predict indoor and outdoor scenes, as well as corresponding entrances and exits.

Description

Technical Field

[0001] This application relates to the field of coarse positioning technology, and in particular to a method, electronic device and chip system for identifying indoor and outdoor scenes. Background Technology

[0002] Various sensors can be installed on electronic devices, and using the data collected by these sensors for localization has become a research hotspot. Localization is divided into precise localization and coarse localization. Precise localization is used to obtain the absolute position of electronic devices, such as latitude and longitude; coarse localization is used to obtain the relative position of electronic devices and their surrounding environment, such as indoor and outdoor scene recognition.

[0003] Even rough localization requires obtaining accurate identification results quickly. Currently, the approach is either to use a large amount of sensor data for complex model calculations to obtain accurate identification results, or to sacrifice accuracy in order to obtain identification results quickly; therefore, it is difficult to obtain accurate identification results quickly. Summary of the Invention

[0004] This application provides a method, electronic device, and chip system for recognizing indoor and outdoor scenes, which can quickly obtain accurate recognition results.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, embodiments of this application provide a method for recognizing indoor and outdoor scenes, including:

[0007] Acquire data corresponding to the user's walking trajectory, wherein the data corresponding to the user's walking trajectory is time-related data collected by multiple sensors on the electronic device carried by the user while walking;

[0008] Based on the data corresponding to the user's walking trajectory, determine the target prediction module used to identify indoor and outdoor scenes;

[0009] The prediction result is obtained based on the data related to the target prediction module in the data corresponding to the walking trajectory;

[0010] Based on the satellite data collected by the satellite navigation system in the data corresponding to the walking trajectory, the prediction result is corrected to obtain the recognition result. The recognition result includes the scene corresponding to each time window in the walking trajectory, and the scene includes indoor scene or outdoor scene.

[0011] In this application, a target prediction module for identifying indoor and outdoor scenes can be determined based on data collected by multiple sensors on an electronic device carried by the user while walking. Then, the corresponding data is input into the corresponding target prediction module to obtain the prediction result of the indoor and outdoor scenes. After obtaining the prediction result, the prediction result can be corrected based on satellite data collected by the satellite navigation system to obtain the recognition result of the indoor and outdoor scenes. By determining different prediction modules under different conditions, both prediction efficiency and prediction accuracy can be taken into account. At the same time, the method of correcting the prediction result with satellite data can improve the prediction accuracy.

[0012] As one implementation of the first aspect, the target prediction module used to identify indoor and outdoor scenes based on the data corresponding to the user's walking trajectory includes:

[0013] The user's state is determined based on the data corresponding to the user's walking trajectory. The user's state includes a moving state or a non-moving state.

[0014] If the user is in motion and the number of samples in the data corresponding to the walking trajectory is less than a first value, then the target prediction module used to identify indoor and outdoor scenes includes: a decision tree classifier, and the sub-target prediction module used in the decision tree classifier includes a joint prediction module, which is used to predict data collected by multiple sensors.

[0015] If the user is in motion, and the number of samples in the data corresponding to the walking trajectory is not less than the first value and less than the second value, then the target prediction module used to identify indoor and outdoor scenes includes: the decision tree classifier, and the sub-target prediction module used in the decision tree classifier includes the ambient light prediction module and the joint prediction module.

[0016] If the user's state is in motion and the number of samples in the data corresponding to the walking trajectory is not less than the second value, then the target prediction module used to identify indoor and outdoor scenes is determined to include at least one of a turning prediction module and a geomagnetic prediction module, as well as the decision tree classifier. The sub-target prediction module used in the decision tree classifier includes at least one of an ambient light prediction module, a satellite prediction module, and a base station prediction module, as well as the joint prediction module.

[0017] If the user is in a non-moving state and the number of samples in the data corresponding to the walking trajectory is less than the third value, then the target prediction module used to identify indoor and outdoor scenes includes: the decision tree classifier, the sub-target prediction module used in the decision tree classifier includes the joint prediction module, and the third value is greater than the first value;

[0018] If the user is in a non-moving state and the number of samples in the data corresponding to the walking trajectory is not less than the third value, then the target prediction module used to identify indoor and outdoor scenes includes: the decision tree classifier, and the sub-target prediction module used in the decision tree classifier includes the ambient light prediction module and the joint prediction module.

[0019] In this embodiment, a basic module for prediction is defined: a joint prediction module. Other prediction models can be determined based on whether the user is in motion and the number of samples. When the user is in motion, modules related to the user's motion behavior can be used, such as a turning prediction module and a geomagnetic prediction module. When there is a large amount of sample data, modules requiring more sample data to achieve accurate prediction results can be used, such as an ambient light prediction module, a satellite prediction module, and a base station prediction module. This approach reduces the power consumption of prediction modules that are less accurate in specific scenarios and avoids these inaccurate predictions affecting the final prediction result, thus improving accuracy while reducing power consumption. Of course, the above classifications are only examples; other classification methods may be used in practical applications.

[0020] As another implementation of the first aspect, determining the user's state based on the data corresponding to the user's walking trajectory includes:

[0021] Based on the data collected by the accelerometer and gyroscope sensors in the data corresponding to the user's walking trajectory, the number of steps taken by the user in the user's walking trajectory is determined.

[0022] The user's status is determined based on the number of steps the user takes in the user's walking trajectory.

[0023] In this application, the number of steps a user takes can be determined based on data collected by an accelerometer and a gyroscope, thereby determining whether the user is in a moving or non-moving state. It should be noted that a non-moving state does not necessarily mean a completely static state. The distinction between moving and non-moving states is used to determine the prediction module; therefore, the distinction is not a strict definition of moving and static states.

[0024] As another implementation of the first aspect, the input data of the joint prediction module includes: data collected by at least two of the following from the data corresponding to the walking trajectory: geomagnetic sensor, accelerometer sensor, gyroscope sensor, satellite navigation module, wireless communication module, and mobile communication module.

[0025] In this application, the settings of the various sub-models in the joint prediction module are different, and the input data may also be different.

[0026] As another implementation of the first aspect, obtaining the prediction result based on data related to the target prediction module from the data corresponding to the walking trajectory includes:

[0027] Based on the sub-target prediction module in the decision tree classifier and the input data corresponding to the sub-target prediction module, the first sub-prediction result corresponding to the decision tree classifier is obtained;

[0028] When the target prediction module includes the turning prediction module, the second sub-prediction result corresponding to the turning prediction module is obtained based on the turning prediction module and the input data corresponding to the turning prediction module.

[0029] When the target prediction module includes the geomagnetic prediction module, the third sub-prediction result corresponding to the geomagnetic prediction module is obtained based on the geomagnetic prediction module and the input data corresponding to the geomagnetic prediction module.

[0030] The prediction result is obtained by fusing the sub-prediction results corresponding to each target prediction module.

[0031] In this application, multiple models can be used to obtain separate sub-prediction results. Then, the sub-prediction results of multiple models are fused to obtain the fused result. This method can make up for the defects of a single prediction model and make the fused prediction result more accurate.

[0032] As another implementation of the first aspect, obtaining the first sub-prediction result corresponding to the decision tree classifier based on the sub-target prediction module in the decision tree classifier and the input data corresponding to the sub-target prediction module includes:

[0033] Based on the prediction outputs and corresponding weights of each sub-target prediction module in the decision tree classifier, the first sub-prediction result corresponding to the decision tree classifier is obtained.

[0034] As another implementation of the first aspect, the fusion of the sub-prediction results corresponding to each target prediction module to obtain the prediction result includes:

[0035] The prediction result is obtained based on the sub-prediction result corresponding to each target prediction module and the weight corresponding to each target prediction module. The sub-prediction result includes the predicted scenario and confidence level, and the weight corresponding to the target prediction module is related to the confidence level in the sub-prediction result corresponding to the target prediction module.

[0036] As another implementation of the first aspect, the calculation method of the second confidence level in the second sub-prediction result includes:

[0037] Based on the data collected by the accelerometer and the gyroscope in the data corresponding to the walking trajectory, the number of turns and the number of non-turns are obtained for each time window in the walking trajectory.

[0038] Based on the number of turns and the number of non-turns corresponding to each time window in the walking trajectory, the second confidence level corresponding to each time window in the walking trajectory is obtained.

[0039] As another implementation of the first aspect, the step of obtaining the number of turns and the number of non-turns corresponding to each time window in the walking trajectory based on the data collected by the accelerometer sensor and the data collected by the gyroscope sensor in the data corresponding to the walking trajectory includes:

[0040] Calculate the average acceleration value of the acceleration collected by the accelerometer in each sub-time period of the first time period, wherein the first time period includes a first time window, and the first time window is any time window in the walking trajectory, and the duration of each sub-time period is less than the duration of the first time window.

[0041] Calculate the average angular velocity of the angular velocities collected by the gyroscope sensor in each sub-time period within the first time period;

[0042] Calculate the projection of the mean angular velocity in each sub-time period of the first time period onto the direction of the mean acceleration to obtain the turning angle corresponding to each sub-time period;

[0043] The number of sub-time periods in which the turning angle is greater than the angle threshold and the turning angle is greater than the turning angle of the previous sub-time period is taken as the number of turns corresponding to the first time window.

[0044] The number of sub-time periods in which the turning angle is not greater than the angle threshold, or the turning angle is not greater than the turning angle of the previous sub-time period, is taken as the number of times the first time window does not turn.

[0045] As another implementation of the first aspect, the center time of the first time period is the same as the center time of the first time window, and the duration of the first time period is greater than or equal to the duration of the first time window.

[0046] In this application, the first time period is set to be larger than the first time window, which is equivalent to using data from a larger time period that includes the first time window as the data for the first time window, which can reduce the interference caused by noisy data.

[0047] As another implementation of the first aspect, obtaining the second confidence level corresponding to each time window in the walking trajectory based on the number of turns and the number of non-turns corresponding to each time window includes:

[0048] according to Calculate the second confidence level for each time window in the walking trajectory;

[0049] Where CI-2 represents the second confidence level, turn_count is the number of turns (if turn_count is greater than 10, then turn_count takes the value 10), noTurn is the number of no turns, and t and a are both penalty factors.

[0050] As another implementation of the first aspect, the calculation method of the third confidence level in the third sub-prediction result includes:

[0051] Based on the data collected by the accelerometer and gyroscope sensors in the data corresponding to the walking trajectory, the number of walking steps corresponding to each time window in the walking trajectory is obtained;

[0052] Based on the data collected by the geomagnetic sensor in the data corresponding to the walking trajectory and the number of walking steps, the number of main peaks of the geomagnetic data corresponding to each time window in the walking trajectory is obtained.

[0053] The third confidence level is obtained based on the number of main peaks of the geomagnetic data corresponding to each time window in the walking trajectory.

[0054] As another implementation of the first aspect, obtaining the third confidence level corresponding to each time window of the walking trajectory based on the number of main peaks of the geomagnetic data corresponding to each time window includes:

[0055] according to Calculate the third confidence level for each time window in the walking trajectory;

[0056] Where CI-3 represents the third confidence level, PeakNum represents the number of main peaks in the geomagnetic data within the time window, Maxsample represents the number of geomagnetic samples within the time window, K represents the size of the time window, and thre represents the threshold for the number of peaks.

[0057] As another implementation of the first aspect, the step of correcting the prediction result based on satellite data collected by the satellite navigation system in the data corresponding to the walking trajectory to obtain the identification result includes:

[0058] The prediction results of each time window in the walking trajectory are viewed in turn. If the prediction result of the previous time window is an outdoor scene and the prediction result of the current time window is an indoor scene, it is determined whether the current time window is in the stage of decreasing satellite number.

[0059] If the current time window is in the middle of the decreasing phase of the number of satellites, the number of satellites before the decreasing phase of the current time window is greater than the first number, the number of satellites after the decreasing phase of the current time window is less than the second number, and the event interval between the current time window and the starting position of the decreasing phase of the current time window is less than the first duration, then no correction is made, and the first number is greater than or equal to the second number.

[0060] If the current time window is in the middle of a phase in which the number of satellites decreases, the number of satellites before the current time window's phase of decrease is greater than a first number, the number of satellites after the current time window's phase of decrease is less than a second number, and the event interval between the current time window and the starting position of the current phase of decrease is not less than a first duration, then the entry point is corrected to the starting position of the current phase of decrease, and the scene in the time window between the starting position of the current phase of decrease and the current time window is corrected to an indoor scene.

[0061] If the current time window is in the middle of the decreasing phase of the number of satellites, and the number of satellites before the decreasing phase of the current time window is greater than the first number, and the number of satellites after the decreasing phase of the current time window is not less than the second number, then the scene of the current time window will be corrected to an outdoor scene.

[0062] If the current time window is in the middle of a phase where the number of satellites is decreasing, and the number of satellites before the current time window is decreasing is no greater than the first number, and the number of satellites after the current time window is decreasing is greater than the second number, then the scene in the time window before the current time window will be corrected to an indoor scene, until the scene in a certain time window is corrected to an indoor scene.

[0063] In this application, the satellite change points corresponding to buildings with large entrances, such as shopping malls and office buildings, are closer to the interior of the building compared to buildings with smaller entrances, such as residential buildings. Therefore, for walking trajectories from outdoors to indoors, if the predicted switching point is in the middle or after the satellite has fully descended, the switching point needs to be corrected to the beginning of the descent. This reduces entrance delay and improves the response speed for switching between indoor and outdoor locations, regardless of the size of the building entrance. On the other hand, if the number of satellites after the predicted entrance point's descent phase is still relatively large, it is equivalent to still being outdoors; therefore, the indoor location after the predicted entrance point needs to be corrected to be outdoors. Conversely, if the number of satellites before the predicted entrance point's descent phase is relatively small, it is equivalent to still being indoors; therefore, the outdoor location before the predicted entrance point needs to be corrected to be indoors. In this way, entrance points with prediction delays can be corrected to more accurate locations, and incorrectly predicted entrance points can be eliminated, improving the accuracy of identification.

[0064] In a second aspect, an electronic device is provided, including a processor for running a computer program stored in a memory to implement the method of any one of the first aspects of this application.

[0065] Thirdly, a chip system is provided, including a processor coupled to a memory, wherein the processor executes a computer program stored in the memory to implement the method of any one of the first aspects of this application.

[0066] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by one or more processors, implements the method of any one of the first aspects of this application.

[0067] Fifthly, embodiments of this application provide a computer program product that, when run on a device, causes the device to execute the method of any one of the first aspects of this application.

[0068] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0069] Figure 1 A schematic diagram of the hardware structure of an electronic device used in the indoor and outdoor scene recognition method provided in the embodiments of this application;

[0070] Figure 2 This application provides a technical architecture diagram of an indoor / outdoor scene recognition method.

[0071] Figure 3 This is a schematic diagram of the data flow of an indoor / outdoor scene recognition method provided in an embodiment of this application;

[0072] Figure 4 A schematic diagram of the data flow of an XGBoost model provided in an embodiment of this application;

[0073] Figure 5 A schematic diagram comparing the number of turns and the number of non-turns in an indoor scene, provided as an embodiment of this application;

[0074] Figure 6 A schematic diagram comparing the number of turns and the number of non-turns in an outdoor scene, provided as an embodiment of this application;

[0075] Figure 7 This application provides a schematic diagram illustrating the change in the number of satellites in an indoor / outdoor scene.

[0076] Figure 8 This is a schematic diagram illustrating the process of determining a prediction module provided in an embodiment of this application;

[0077] Figure 9 A schematic diagram illustrating a process for calculating confidence level using a turning prediction module, provided as an embodiment of this application;

[0078] Figure 10 A schematic diagram illustrating a process for calculating confidence level using a geomagnetic prediction module, provided as an embodiment of this application;

[0079] Figure 11 A schematic diagram of satellite change curves when moving from indoors to outdoors and from outdoors to indoors, provided as an embodiment of this application;

[0080] Figure 12 A schematic diagram illustrating the relationship between predicted entry points and the number of satellites provided for embodiments of this application;

[0081] Figure 13 This is a schematic diagram illustrating the process of modifying the entry point as provided in an embodiment of this application. Detailed Implementation

[0082] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0083] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0084] It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between the associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0085] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," "fourth," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0086] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0087] The indoor / outdoor scene recognition method provided in this application can be applied to the following electronic devices: tablet computers, mobile phones, wearable devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. This application does not limit the specific type of electronic device.

[0088] Figure 1 A schematic diagram of an electronic device is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a touch sensor 180K, an ambient light sensor 180L, etc.

[0089] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0090] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a sensor coprocessor (SCP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. For example, processor 110 is used to execute the calibration method for the ambient light sensor in the embodiments of this application.

[0091] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0092] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0093] USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc. USB interface 130 can be used to connect a charger to charge electronic device 100, and it can also be used for data transfer between electronic device 100 and peripheral devices.

[0094] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0095] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0096] Internal memory 121 can be used to store executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function (such as sound playback, image playback, etc.).

[0097] In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0098] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130.

[0099] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0100] In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may also be located in the same device.

[0101] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0102] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0103] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1.

[0104] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0105] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0106] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0107] The audio module 170 is used to convert digital audio signals into analog audio signals for output, and also to convert analog audio inputs into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0108] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0109] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0110] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to monitoring voice information, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc. For example, microphone 170C can be used to collect voice information involved in the embodiments of this application.

[0111] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0112] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When a force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch operation intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A.

[0113] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0114] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0115] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0116] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback.

[0117] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0118] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0119] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0120] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0121] This application does not specifically limit the structure of the execution subject of the indoor / outdoor scene recognition method. As long as it can communicate according to the indoor / outdoor scene recognition method provided by this application by running code that records the recognition of an indoor / outdoor scene according to this application, it is acceptable. For example, the execution subject of the indoor / outdoor scene recognition method provided by this application can be a functional module in an electronic device capable of calling and executing programs, or a communication device applied in an electronic device, such as a chip.

[0122] As electronic devices become increasingly powerful, their user base continues to grow. The portability of small electronic devices makes them an essential part of daily life for many. Simultaneously, location tracking using data collected by sensors on these devices has become a research hotspot, with positioning categorized into precise and coarse positioning. However, this distinction isn't based on positioning accuracy, but rather on the application scenario.

[0123] Precise positioning can be achieved by using latitude and longitude calculated from Bluetooth, Wi-Fi, or satellite systems on electronic devices, thereby obtaining the absolute location or movement trajectory of the electronic device or the user carrying it.

[0124] Rough positioning can utilize various sensors installed on electronic devices to detect the current environment of the electronic device or the user carrying the electronic device, thereby determining the relative positional relationship between the electronic device (or the user carrying the electronic device) and its surrounding environment.

[0125] As an application scenario for coarse positioning, it uses various sensors on electronic devices to detect whether the electronic device or the user carrying the electronic device is indoors or outdoors. It can also detect the transition points between indoors and outdoors, or vice versa. Of course, even coarse positioning requires fast and accurate detection results.

[0126] This application provides an application scenario for coarse positioning. As a user moves around carrying an electronic device, sensors on the device can collect various data. Based on this data, the device can determine whether it (or the user carrying the device) is indoors or outdoors at various points in time (or time windows) during the user's movement. It can also determine the exit point from indoors to outdoors or the entrance point from outdoors to indoors.

[0127] As one application scenario for rough positioning, data collected by sensors on the phone is used to roughly determine whether the phone is indoors or outdoors, and then the screen brightness, volume, and other settings are adjusted accordingly. For example, when indoors, the volume is adjusted to a lower level and the screen brightness to a lower level; when outdoors, the volume is adjusted to a higher level and the screen brightness to a higher level. To adapt to the environment promptly, it is also necessary to accurately determine the transition points between indoor and outdoor scenes (entrances and exits), so that the screen brightness and volume can be adjusted to match the specified values ​​in a timely manner.

[0128] Another scenario for rough positioning involves using data collected by sensors on the phone to roughly determine whether the phone is indoors or outdoors. This allows for adjustments to the on / off status of certain modules on the phone based on the current environment. For example, when indoors, the phone's GPS module is turned off, and other sensors are used for positioning; when outdoors, the GPS module is turned on, and positioning is achieved using the phone's GPS. When moving from indoors to outdoors, to adapt to the environment quickly, the transition point (entrance) needs to be accurately determined, allowing for timely adjustments to the GPS module's status.

[0129] Of course, this application focuses on describing how to accurately identify indoor and outdoor scenes and accurately identify entrances and exits, and does not limit the application scenarios of the identification results of this application. The model provided in this application is used to improve the accuracy of indoor and outdoor scene identification, and does not mean that using the indoor and outdoor scene identification method provided in this application will necessarily obtain completely accurate identification results.

[0130] The electronic device provided in this application requires the following types of sensors: basic sensors, navigation and positioning modules, wireless communication modules, and mobile communication modules.

[0131] Basic sensors include ambient light sensors, accelerometers, gyroscopes, and geomagnetic sensors. Of course, in practical applications, basic sensors may include only some of the sensors listed above, or other sensors beyond those listed, such as barometric pressure sensors and temperature sensors.

[0132] The navigation and positioning module is used to receive signals from a Global Navigation Satellite System (GNSS). The GNSS can be any positioning system, such as GPS (Global Positioning System), BeiDou Navigation Satellite System, etc. Of course, the GNSS can also be a regional navigation system; this application embodiment does not impose any limitations on this. This module can be referred to as the GNSS module.

[0133] The wireless communication module is used to receive signals from wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks); in practical applications, this wireless communication module can also be a module that receives signals from other wireless local area networks, such as a Bluetooth module. This application uses a Wi-Fi module as an example of a wireless communication module.

[0134] The mobile communication module is used to receive signals from nearby base stations. Of course, the base station is not limited to large base stations, but can also be a small base station (CELL) installed inside a building.

[0135] The following describes how to obtain indoor and outdoor scene recognition results using data collected from the various sensors set on the electronic device.

[0136] Reference Figure 2 This is a technical architecture diagram of the indoor and outdoor scene recognition method provided in the embodiments of this application.

[0137] The technical architecture diagram includes a data layer, a data processing layer, a core algorithm layer, and an output layer.

[0138] The data layer includes raw data collected by various sensors. Examples include the signal strength of wireless signals received by the Wi-Fi module; the number of satellites observed and their signal strength by the GNSS module; the received power of signals received from base stations by the mobile communication module, the switching status of connected base stations, and the connection time with base stations; the Earth's magnetic field received by the geomagnetic sensor; the angular velocity collected by the gyroscope sensor; the acceleration collected by the accelerometer sensor; and the ambient light intensity collected by the ambient light sensor.

[0139] The data processing layer is used to preprocess and extract features from the raw data collected by each sensor.

[0140] The data processing layer includes a data alignment module, a data completion module, and a feature extraction module.

[0141] The data alignment module is used to perform time alignment on the raw data collected by each sensor; the data completion module is used to fill in missing items in the raw data collected by each sensor.

[0142] In practice, the time periods for acquiring raw data from different sensors may vary slightly. Therefore, the alignment and padding method does not align the raw data acquired by each sensor in sequence, but rather align them based on the absolute time of acquisition. Of course, some sensors may have missing data points in their raw data, or the aligned sensor data may have unbalanced sample data. In such cases, data padding can be performed. The data alignment and padding methods in this application are for illustrative purposes only; in actual applications, the specific alignment and padding methods used are not limited.

[0143] After preprocessing the raw data using the data alignment and data completion modules, the feature extraction module can extract features from the preprocessed raw data to obtain feature data related to indoor and outdoor scenes. In this embodiment, 198-dimensional features can be obtained after feature extraction. Of course, in practical applications, the specific number of feature dimensions used can be set according to the actual situation.

[0144] The core algorithm layer is used to obtain indoor and outdoor scene recognition results and entrance / exit recognition results based on the data processed by the data processing layer and the corresponding algorithms.

[0145] The core algorithm layer includes: decision tree classifier (XGBoost prediction module), geomagnetic prediction module, turning prediction module, fusion module, and correction module.

[0146] The decision tree classifier comprises several sub-modules: a joint prediction module, a GNSS prediction module, a base station prediction module, and an ambient light prediction module. The first prediction result is obtained by combining the predictions from these multiple modules.

[0147] The input to the joint prediction module includes data collected by accelerometers, gyroscopes, GNSS sensors, wireless communication modules, and mobile communication modules.

[0148] The input to the GNSS prediction module includes data acquired by GNSS sensors.

[0149] The input to the base station prediction module includes data collected by the mobile communication module.

[0150] The input to the ambient light prediction module includes data collected by the ambient light sensor.

[0151] Of course, in practical applications, the sub-modules included in the decision tree classifier can be added or removed according to the actual situation, and the input of the joint prediction module can also be added or removed according to the actual situation.

[0152] The decision tree classifier takes as input the 198-dimensional feature data extracted by the feature extraction module and outputs a first prediction result. This first prediction result is used to represent the indoor / outdoor identification results at various times along the user's walking trajectory.

[0153] The geomagnetic prediction module is used to identify indoor and outdoor scenes based on data collected by geomagnetic sensors and accelerometers (both of which are preprocessed data), and obtain indoor and outdoor identification results (second prediction results) at various times.

[0154] The turning prediction module is used to identify indoor and outdoor scenes based on data collected by the accelerometer and gyroscope sensors (both of which are preprocessed data), and obtain indoor and outdoor identification results at various times (the third prediction result).

[0155] The indoor / outdoor recognition results described above can be presented as a predicted scene (indoor or outdoor), or as a predicted scene (indoor or outdoor) and a confidence level. For example, a predicted scene of 0.8 indicates an indoor scene with an 80% confidence level. The fusion module can combine the three predicted scenes to obtain a fused prediction result. In practical applications, during fusion, the confidence level of the predicted scene can be used to determine the weights of the predicted scenes for each prediction model.

[0156] The correction module is used to correct the fused prediction results to obtain the corrected indoor and outdoor recognition results and the indoor and outdoor switching points.

[0157] The output layer includes an indoor / outdoor scene recognition result output module and an entrance / exit recognition result output module.

[0158] The indoor / outdoor scene recognition result output module is used to output the corrected indoor / outdoor recognition results; the entrance / exit recognition result output module is used to output the corrected indoor / outdoor switching points.

[0159] To better understand the data flow of the data collected by each sensor, refer to Figure 3 The data flow diagram shown.

[0160] Raw data collected by the ambient light sensor, Wi-Fi module, GNSS module, mobile communication module, gyroscope sensor, accelerometer sensor, and geomagnetic sensor are all preprocessed before being input into the feature extraction module. To clearly illustrate the data flow of the raw data collected by each sensor, Figure 3 The data flow diagram shown omits the data preprocessing process.

[0161] After the feature extraction module extracts features from the received data, it inputs the resulting feature data into the XGBoost module.

[0162] Reference Figure 4 This is a data flow diagram within the XGBoost module.

[0163] The input to the joint prediction module includes feature data corresponding to data collected by geomagnetic sensors, accelerometers, gyroscopes, GNSS sensors, wireless communication modules (Wi-Fi modules), and mobile communication modules. Of course, in practical applications, feature data corresponding to data collected by some sensors may also be included; this embodiment does not impose such limitations.

[0164] The input to the GNSS prediction module includes feature data corresponding to the data collected by the GNSS sensor.

[0165] The input to the base station prediction module includes feature data corresponding to the data collected by the mobile communication module.

[0166] The input to the ambient light prediction module includes feature data corresponding to the data collected by the ambient light sensor.

[0167] The raw data collected by the gyroscope sensor and the raw data collected by the accelerometer sensor are preprocessed and then input into the turning prediction module.

[0168] The raw data collected by the accelerometer and the geomagnetic sensor are preprocessed and then input into the geomagnetic prediction module.

[0169] The first prediction result from the XGBoost prediction module, the second prediction result from the turning prediction module, and the third prediction result from the geomagnetic prediction module are all input into the fusion module.

[0170] The fusion module inputs the fused prediction results into the correction module.

[0171] The correction module will output the corrected recognition results separately.

[0172] After describing the technical architecture and data flow provided in the embodiments of this application, the method for identifying indoor and outdoor scenes provided in the embodiments of this application will be described below.

[0173] As mentioned earlier, each sensor can collect raw data. In practical applications, the time period for each sensor to collect raw data may vary slightly.

[0174] This application embodiment defines a sample as the sampling data within a time window. When the sensor's acquisition period is greater than this time window, there will be no sampling data from the sensor within one or more time windows. In practical applications, the sampling data from the sensor within this time window can be set to empty or the sampling data from the previous time window can be reused. When the sensor's sampling period is less than this time window, theoretically, there will be sampling data from the sensor within each time window.

[0175] In some cases, where it's necessary to acquire changes in sampled data from a sensor within a time window, the sensor's sampling period must be much shorter than that time window. This allows the sensor to collect multiple data points within the time window, and the changes in these multiple data points can serve as feature data for the samples corresponding to that time window. The following tables list the feature data for these situations.

[0176] In this embodiment of the application, a longer time window may affect the accuracy of recognition; a shorter time window will increase the frequency of recognizing indoor and outdoor scenes, resulting in a larger computational load and higher power consumption.

[0177] In this embodiment, the time window can be set to 1 second. Of course, in practical applications, the time window can be set to other time values ​​based on the hardware capabilities of the electronic device itself, while also balancing accuracy and power consumption. This embodiment does not limit this.

[0178] In a time window using 1 second as an example, if a sensor has a sampling frequency of 100 Hz, data will be collected at 100 time points within that time window. The data collected within a time window constitutes a data frame. A data frame is considered a data sample.

[0179] In addition, embodiments of this application can set the time window to a sliding window with 50% data overlap, that is, there is 50% time coverage (50% data overlap) between two adjacent time windows.

[0180] As an example, the first time window is from second 0 to second 1, and the first data sample is the data collected from second 0 to second 1; the second time window is from second 0.5 to second 1.5, and the second data sample is the data collected from second 0.5 to second 1.5; the third time window is from second 1 to second 2, and the third data sample is the data collected from second 1 to second 2; and so on.

[0181] The data collected by each sensor within this time window, and the feature data obtained based on the data collected within this time window, will be described below.

[0182] An ambient light sensor is used to collect ambient light intensity. With a time window of 1 second and a sampling frequency of 50 Hz (for example only), the data frame corresponding to this time window will include the ambient light intensity at 20 moments.

[0183] In this embodiment, the data collected by the ambient light sensor at 20 moments within a time window can yield statistical characteristics. These statistical characteristics include: mean, median, maximum, minimum, range, variance, standard deviation, quartiles, skewness, and kurtosis. The statistical characteristics corresponding to the data collected by the ambient light sensor within a time window have 10 dimensions. These 10 dimensions of statistical characteristics can reflect the ambient light intensity of the environment in which the electronic device is currently located.

[0184] Typically, the ambient light intensity values ​​differ between indoors and outdoors. Tests have shown that when electronic devices are outdoors, the ambient light sensor on the device usually measures a higher ambient light intensity value, while when the device is indoors, the measured ambient light intensity value is lower.

[0185] In this embodiment, the threshold value for the ambient light intensity of the indoor and outdoor scenes can be set to 2000 Lux by repeatedly measuring the ambient light intensity values ​​of both scenes. That is, when the ambient light intensity detected by the ambient light sensor on the electronic device is less than 2000 Lux, the electronic device is determined to be in an indoor scene. When the ambient light intensity detected by the ambient light sensor is greater than or equal to 2000 Lux, the electronic device is determined to be in an outdoor scene.

[0186] Of course, in some scenarios, such as when the ambient light sensor on an electronic device is blocked, the ambient light intensity collected by the ambient light sensor is usually very small. Therefore, a very small value (e.g., 50 Lux) can be set. If the measured ambient light intensity value is less than this very small value, it is determined that the ambient light sensor is blocked. In the scenario where the ambient light sensor is blocked, it is no longer necessary to determine whether it is an indoor or outdoor scene.

[0187] In summary, in this embodiment of the application, it can be set that when the ambient light intensity collected by the ambient light sensor is between 50 Lux and 2000 Lux, the electronic device is determined to be in an indoor scene; when the ambient light intensity collected by the ambient light sensor is greater than or equal to 2000 Lux, the electronic device is determined to be in an outdoor scene.

[0188] Those skilled in the art will understand that 50 Lux and 2000 Lux are for illustrative purposes only. In practical applications, other values ​​can be determined as critical values ​​based on the measurement characteristics of the ambient light sensor itself.

[0189] Of course, the above example is only one way to determine whether a scene is indoors or outdoors. In practical applications, all 10 statistical features can be used as parameters to determine whether a scene is indoors or outdoors. That is, the 10 features of ambient light intensity listed above can be used to determine whether the current scene is indoors or outdoors.

[0190] In practical applications, the ambient light prediction module in the XGBoost prediction module can predict indoor and outdoor scenes based on 10 dimensions of ambient light intensity feature data.

[0191] Accelerometers are commonly used to measure the acceleration and related parameters of electronic devices. The characteristic data corresponding to the data obtained by the accelerometer can be found in Table 1.

[0192] Table 1. Feature data corresponding to the accelerometer.

[0193]

[0194] Gyroscope sensors are commonly used to measure the angular velocity and related parameters of electronic devices. The characteristic data corresponding to the data obtained from gyroscope measurements can be found in Table 2.

[0195] Table 2. Feature data corresponding to gyroscope sensors

[0196]

[0197]

[0198] The parameters measured by accelerometers and gyroscopes can reveal the motion of a user carrying an electronic device. This motion can include turning; for example, if the angle detected by the electronic device exceeds a certain threshold, a turn is confirmed; otherwise, no turn is determined. A time window can be divided into k time intervals, yielding the number of turns (also called the number of turns) and the number of no turns (also called the number of no turns) over k consecutive time intervals. The number of turns and no turns over k consecutive time intervals can also be used as feature data in 198 dimensions. See Table 3 for details.

[0199] Table 3 shows the number of turns and non-turns obtained from parameters measured by the accelerometer and gyroscope sensors.

[0200]

[0201] Actual measurements revealed that when users are carrying electronic devices indoors, the number of turns they make is significantly higher than when they are carrying electronic devices outdoors due to the relatively small space, numerous items, fragmented zoning, and dense population.

[0202] Reference Figure 5 This is a schematic diagram comparing the number of turns in indoor and outdoor scenes obtained from tests in an embodiment of this application. (Refer to...) Figure 5 In the figure (a), the percentage of different number of turns in the indoor scene is represented. We can count the number of turns corresponding to each time window (e.g., 1 second) during a user's movement with electronic devices indoors (e.g., 35 seconds), resulting in 35 number of turns; then determine the number of time windows corresponding to each number of turns (e.g., the number of time windows with 1 turn, the number of time windows with 2 turns), and finally, obtain the percentage of the number of time windows corresponding to each number of turns out of the total number of time windows (35). Figure 5 (a) shows the percentage of different number of turns in an indoor scene.

[0203] It can be obtained in the same way. Figure 5 The percentage of time windows corresponding to each number of turns in the outdoor scene shown in (b) is the total number of time windows.

[0204] The comparison reveals that the number of turns indoors ranges from 1 to 7. Outdoors, the number of turns is 0 in 50% of cases, with the majority of turns falling between 0 and 2. This demonstrates that indoor turns involve more turns than outdoor turns.

[0205] Reference Figure 6 This is a schematic diagram comparing the number of times no turns were made in indoor and outdoor scenes, obtained from tests in embodiments of this application. Figure 6 The method shown in the diagram can be referred to Figure 5 The process obtained from (a) in the text. (Refer to...) Figure 6 As shown in (a) above, in indoor scenes, the number of non-turning times is relatively dispersed, ranging from 3 to 13 times. (See reference...) Figure 6 As shown in (b), in outdoor scenes, the number of non-turns is concentrated between 9 and 15. Relatively speaking, there are more non-turns outdoors than turns indoors.

[0206] As can be understood from the above description, indoor and outdoor scenes can be predicted based on the number of turns and the number of times they do not turn.

[0207] Geomagnetic sensors are used to measure the Earth's magnetic force. Based on the measured Earth's magnetic force, geomagnetic changes can be obtained. The characteristic data corresponding to the data measured by the geomagnetic sensor can be referred to in Table 4.

[0208] Table 4. Characteristic data corresponding to the geomagnetic sensor

[0209]

[0210] Actual measurements revealed that when users are outdoors with electronic devices, the environment is relatively open, surrounding buildings are relatively far away, overhead is usually unobstructed, and geomagnetic disturbances are minimal. Conversely, when users are indoors with electronic devices, geomagnetic disturbances are greater. Therefore, in indoor and outdoor scenarios, the peaks, troughs, variance, and standard deviation of geomagnetic data sequences corresponding to k consecutive time periods will exhibit different characteristics. For example, indoor scenarios tend to have more peaks and troughs than outdoor scenarios, and the variance is also larger indoors compared to outdoors.

[0211] The parameters in Table 4 represent the number of peaks, the number of troughs, the variance of the geomagnetic sequence, and the standard deviation of the geomagnetic sequence over k consecutive time periods.

[0212] The data corresponding to each parameter in Tables 1 to 4 above, as well as the data corresponding to the 10-dimensional features of the ambient light intensity measured by the ambient light sensor, are the feature data obtained by the basic sensor measurement in this application embodiment.

[0213] A GNSS module can measure the azimuth distribution of satellites and the strength of satellite signals.

[0214] In practical applications, reinforced concrete and walls can block satellite signals. Satellite signals can penetrate glass curtain walls and windows. Therefore, deep indoors, surrounded mostly by reinforced concrete and walls, electronic devices have difficulty receiving satellite signals. In shallower indoor areas, surrounded by reinforced concrete and walls in some areas and glass curtain walls and windows in others, the azimuth angle of satellite signals received by electronic devices is limited. In semi-outdoor areas, with taller buildings, overpasses, etc., the area exposed to the sky is larger than in shallower indoor areas, and electronic devices can receive satellite signals after multipath reflection. In open outdoor areas, the GNSS module on electronic devices can receive relatively strong satellite signals.

[0215] Based on the above analysis, the horizontal azimuth angle can be divided into 36 sectors, and the vertical azimuth angle into 9 sectors, which is equivalent to dividing the three-dimensional space where the electronic device is located into 324 partitions. The GNSS sensor on the electronic device can acquire satellite data for these partitions, such as the number of visible satellites (the number of observed satellites), the proportion of partitions with visible satellites, and the partition with the largest number of satellites.

[0216] In this application embodiment, the satellite data for a single moment is recorded as a single sample feature.

[0217] In practical applications, when a user carries electronic devices from outdoors to indoors, the number of satellite signals detected by the GNSS module gradually decreases. Similarly, when a user carries electronic devices from indoors to outdoors, the number of satellite signals detected by the GNSS module gradually increases.

[0218] Reference Figure 7 This diagram illustrates the change in the number of satellites observed by the GNSS module when moving from indoors to outdoors, as provided in an embodiment of this application. It can be understood that the number of observed satellites is zero deep indoors, increases as the indoor depth increases, and continues to increase as one moves outdoors. The number of observed satellites becomes relatively stable when entering a more open outdoor space. Therefore, this embodiment of the application can use the relevant information of the satellites observed by the GNSS module as feature data to predict indoor and outdoor scenes, or to correct indoor and outdoor scenes, etc.

[0219] Of course, in practical applications, other data may also change when users move between indoors and outdoors while carrying electronic devices. For example, the number of visible satellites may change over multiple consecutive time periods, the percentage of visible satellites that have changed over multiple consecutive time periods, the percentage of satellites with an increased carrier noise ratio over multiple consecutive time periods, the percentage of satellites with a unchanged carrier noise ratio over multiple consecutive time periods, and the percentage of satellites with a decreased carrier noise ratio over multiple consecutive time periods.

[0220] This application embodiment can statistically analyze the changes in satellite data over multiple consecutive time periods, identifying features over k consecutive time periods. These features are denoted as multiple consecutive sample features, enabling the detection and identification of indoor and outdoor scenes based on these multiple consecutive sample features.

[0221] As an example, the calculation process for the percentage of satellites whose carrier noise ratio increases (or decreases, or remains unchanged) over multiple consecutive time periods is as follows:

[0222] f(x,y)=sign(cnr i x -cnr i y | i = 1, 2, ..., n;

[0223] Where f(x,y) represents the signal strength change of the same satellite between time y and time x, cnr i x cnr represents the satellite signal strength of satellite i at time x. i yThis represents the satellite signal strength of satellite i at time y, where n is the total number of satellites. Time y and time x are two consecutive sampling times, and the period from time y to time x is one time interval.

[0224]

[0225]

[0226] P up =P change -P down ;

[0227] P hold =1-P down -P up ;

[0228] Among them, P down P represents the percentage of visible satellites whose carrier noise ratio decreases over k consecutive time periods. change P represents the percentage of visible satellites whose carrier noise ratio changes over k consecutive time periods. up P represents the percentage of visible satellites whose carrier noise ratio increases over k consecutive time periods. hold This represents the percentage of visible satellites whose carrier noise ratio remains constant over k consecutive time periods.

[0229] The calculation process for other parameters will not be illustrated one by one. The single sample features and multiple consecutive sample features corresponding to the GNSS module can be referred to in Table 5.

[0230] Table 5. Feature data corresponding to the data collected by the GNSS module.

[0231]

[0232] It is understood that in this embodiment of the application, it is not necessary to obtain all the data collected by GNSS for calculating latitude and longitude, but only some information such as satellite number, signal-to-noise ratio, elevation angle and azimuth angle are needed, which reduces the amount of identification calculation and improves efficiency.

[0233] The wireless communication module in this embodiment is a Wi-Fi module, used to analyze the wireless signals received by the electronic device in indoor and outdoor scenarios.

[0234] Wireless routers are usually installed indoors. Therefore, when users are indoors with their electronic devices, the Wi-Fi signals received by the devices are stronger and more numerous. Conversely, when users are outdoors with their electronic devices, the Wi-Fi signals received by the devices are weaker and fewer.

[0235] Of course, the strength and quantity of signals may change when a user moves between indoors and outdoors while carrying electronic devices.

[0236] Based on the above analysis, the features obtained from the data collected by the Wi-Fi module can include single sample features and multiple consecutive sample features. Table 6 shows the single sample features and multiple consecutive sample features obtained by the Wi-Fi module.

[0237] Table 6. Feature data corresponding to the data collected by the Wi-Fi module.

[0238]

[0239] Operators typically build numerous base stations, and the mobile communication modules in electronic devices communicate with these nearby base stations. However, within large buildings, the signal received by the mobile communication modules of electronic devices may be obstructed by building walls, resulting in poor signal strength. Therefore, many operators or building structures install small base stations indoors. These small base stations have limited coverage, and their signal strength varies rapidly with distance. Consequently, when a user moves indoors with their electronic device, the signal received by the mobile communication module from these small base stations changes quickly. Of course, if the electronic device is within the coverage area of ​​multiple small base stations, it will switch to the one with the strongest signal. Therefore, the connection time between the electronic device and the small base station is usually short when the user moves indoors with their electronic device.

[0240] Conversely, when a user is outdoors with an electronic device, because large base stations are sparsely distributed outdoors, the electronic device will connect more stably to the nearest base station. The signal strength received by the electronic device from the base station changes more slowly, and the connection time to a base station is also longer. Therefore, in this embodiment, the base station data collected by the mobile communication module can also be used as feature data for indoor and outdoor scene prediction.

[0241] In this application embodiment, the Reference Signal Receiving Power (RSRP) can be set as a parameter of signal strength.

[0242] Similarly, the parameters obtained by the mobile communication module on an electronic device can include single sample features and multiple consecutive sample features. See Table 7 for details.

[0243] Table 7 Feature data corresponding to the data collected by the mobile communication module

[0244]

[0245] The four types of sensors listed above can yield 198 parameters. These 198 parameters can be used for subsequent indoor and outdoor scene recognition.

[0246] It should be noted that the 198-dimensional feature data is for illustrative purposes only. In practical applications, other dimensions of feature data or different feature data can be obtained based on the type and number of sensors.

[0247] The tables above have obtained the feature data corresponding to each sensor; the feature data obtained after feature extraction from the raw data collected by each sensor can be input into the XGBoost model to obtain the first prediction result.

[0248] The prediction process of the XGBoost model will be described below.

[0249] As mentioned above, in this embodiment of the application, the feature data of each sensor extracted by the feature extraction module can be input into the XGBoost model to identify indoor and outdoor scenes and output the indoor and outdoor scene identification results.

[0250] The input to the XGBoost model includes the feature matrix corresponding to the feature data extracted from the raw data collected by various sensors on the electronic device (corresponding to each time or location). Taking a walking trajectory that includes n time windows as an example, the input is a feature matrix of 198 multiplied by n.

[0251] The output of the XGBoost model includes the prediction results of the electronic device in each time window and the corresponding probability values ​​of the prediction results.

[0252] See Table 8 for the parameters in the XGBoost model and the parameters during the training process.

[0253]

[0254]

[0255] Before using the XGBoost model for prediction, it needs to be trained. This application embodiment sets up three modes for the model: Train, SaveModel, and Test.

[0256] Before performing any training, a dataset for training is required. The following describes the method for obtaining the dataset provided in the embodiments of this application.

[0257] This application embodiment can acquire raw data collected by various sensors on a user's electronic device while the user moves around in multiple locations. These multiple locations may include schools, office buildings, shopping malls, and residential buildings, etc. The number of walking trajectories in these multiple locations exceeds 250, and the raw data collected by the various sensors on the electronic device may reach up to 80,000. Each raw data point is timestamped, representing the time when the data was collected. A feature matrix can be obtained for each walking trajectory. Furthermore, in practical applications, the time length corresponding to each walking trajectory can be set. For example, if the time length of a walking trajectory is too long, it can be divided into multiple smaller walking trajectories. This application embodiment does not limit the specific form.

[0258] A behavior label is set for the data corresponding to each walking trajectory (or sub-walking trajectory). The behavior label is the scene (indoor or outdoor) corresponding to each time window (each sample). Of course, the dividing point between outdoor and indoor scenes in the behavior label is the switching point between indoor and outdoor. The feature data corresponding to the walking trajectories with these behavior labels constitute the dataset. The specific form of the behavior label is not limited in this embodiment.

[0259] In Train mode, the feature data corresponding to each walking trajectory in the dataset is input into the XGBoost model. A five-fold cross-validation algorithm is used to train the XGBoost model. During training, the dataset is trained in its original order. This method allows for greater device heterogeneity and spatiotemporal differences between the four-fold training data and the single-fold validation data, enabling more accurate observation of the model's generalization ability. The five-fold cross-validation algorithm divides the dataset into five piles; one pile is selected as the test set (serving as the single-fold validation data), and the other four piles are used as the training set (serving as the four-fold training data). This process is repeated five times, with the test set selected different each time. For example, the first pile might be selected as the test set in the first iteration, the second pile in the second iteration, the third pile in the third iteration, and so on.

[0260] In SaveModel mode, the feature data corresponding to each walking trajectory that has been trained in the dataset can be input into the model in the form of a feature matrix to obtain the trained model; the trained model can then be used to predict indoor and outdoor scenes.

[0261] In Test mode, the feature data corresponding to the user's walking trajectory with electronic devices can be input into the trained XGBoost model in the form of a feature matrix to obtain the prediction results of the XGBoost model.

[0262] In practical applications, the XGBoost module includes multiple sub-models, and each sub-model can be trained in the manner described above. The input data for training is as described above. Figure 4 The relevant descriptions in the data flow diagram are shown below and will not be repeated here. It should be noted that in practical applications, the data features corresponding to the data collected by each sensor can be input into the XGBoost model to obtain a first prediction result. This first prediction result can be the fusion of the outputs of each sub-module in the XGBoost model. Each sub-module has different weights, and the first prediction result of the XGBoost model can be obtained based on the output of each sub-module and its corresponding weight. In practical applications, only some sub-modules of the XGBoost model can be used for prediction.

[0263] As an example, refer to Figure 8 The user's movement trajectory data is divided into multiple categories.

[0264] (1) If the user is in motion and the number of samples in the data corresponding to the walking trajectory (data in each time window is considered as one sample) is less than a first value (e.g., 30), then the sub-target prediction module used in the decision tree classifier (XGBoost) includes a joint prediction module, which is used to predict data collected by multiple sensors.

[0265] (2) If the user is in motion and the number of samples in the data corresponding to the walking trajectory is not less than the first value and less than the second value (e.g., 120), then the sub-target prediction module used in the decision tree classifier includes the ambient light prediction module and the joint prediction module.

[0266] (3) If the user's state is in motion and the number of samples in the data corresponding to the walking trajectory is not less than the second value, then the sub-target prediction module used in the decision tree classifier includes at least one (or at least two, or all) of the ambient light prediction module, satellite prediction module and base station prediction module, as well as the joint prediction module; of course, it can also be each sub-model in XGBoost.

[0267] (4) If the user is in a non-moving state and the number of samples in the data corresponding to the walking trajectory is less than the third value, the sub-target prediction module used in the decision tree classifier includes the joint prediction module, the third value is greater than the first value, and the third value can be the same as or different from the second value.

[0268] (5) If the user is in a non-moving state and the number of samples in the data corresponding to the walking trajectory is not less than the third value, then the sub-target prediction module used in the decision tree classifier includes the ambient light prediction module and the joint prediction module.

[0269] In practical applications, a user's state can be determined by data collected from an accelerometer and a gyroscope. For example, the number of steps taken in a user's walking trajectory can be determined, and then the user's state can be determined based on the number of steps. As an example, when the number of steps is greater than a certain value, it indicates a moving state; when the number of steps is less than a certain value, it indicates a non-moving state.

[0270] In another embodiment of this application, a walking trajectory can be divided into multiple sub-trajectories based on the number of steps taken within each time window. This divides a large walking trajectory into sub-trajectories in motion and those in non-motion states, and different sub-modules are used for prediction of the sub-trajectories in different states. Finally, the prediction results of the used sub-modules are fused with their weights to obtain the first prediction result of XGBoost. A walking trajectory can yield first prediction results for multiple sub-time periods. Concatenating the first prediction results of multiple sub-time periods yields the prediction result of the original walking trajectory.

[0271] Of course, in practical applications, a walking trajectory can also be divided into multiple sub-walking trajectories according to a fixed duration (greater than or equal to one time window). For each sub-walking trajectory, different sub-modules in the XGBoost model are used for prediction according to the situations listed above. Finally, the prediction results of multiple sub-modules are merged to obtain the first prediction result. Finally, the first prediction results corresponding to multiple sub-walking trajectories are spliced ​​together to obtain the prediction result of the original walking trajectory.

[0272] The embodiments in this application do not limit the specific implementation process, but are only used as examples to illustrate the division of the above five situations.

[0273] In addition, in practical applications, each sub-module in the XGBoost model can make predictions based on the input data to obtain the prediction results of each sub-module. Finally, based on the above 5 cases, the corresponding outputs are selected for fusion to obtain the first prediction result.

[0274] The following describes the process by which the turning prediction module obtains confidence scores when predicting indoor and outdoor scenes. As mentioned earlier, the turning frequency, i.e., the number of turns per unit effective motion time, can be obtained from the accelerometer and gyroscope. Effective motion time represents the time interval during which a person is in motion within a certain period.

[0275] In practical applications, the turning prediction module can be used to predict indoor and outdoor scenes while the user is moving. Whether the user is moving or not can be determined by an accelerometer.

[0276] The confidence level is calculated based on the turning frequency within the user's effective movement time and a turning frequency threshold. The turning frequency threshold can be determined by statistically analyzing the number of turns made by multiple users within a unit of effective movement event in multiple indoor scenarios.

[0277] See Figure 9 When obtaining the confidence level of the scene predicted by the turning prediction module, we can first calculate the average value of the acceleration obtained by the accelerometer and the average value of the angular velocity obtained by the gyroscope for each sub-time period (less than time window a, usually a time window can include multiple sub-time periods) within a time period A (e.g., 3 seconds).

[0278] Then, the turning angle for each sub-time period (each sample in the current stage) is calculated sequentially: when calculating the turning angle for each sub-time period, the projection of the angular velocity of the gyroscope in that sub-time period onto the acceleration vector of that sub-time period can be used as the real-time angle value, which is used as the user's turning angle.

[0279] Secondly, when the turning angle is greater than the turning angle threshold and the turning angle is on an increasing trend (e.g., greater than the turning angle of the previous sub-time period), it is considered that a turn has occurred; otherwise (if the above conditions are not met), it is considered that no turn has occurred.

[0280] In practical applications, we can sequentially check whether the turning angle of each sub-time period (each sample in the current stage) (the turning angle of the i-th time period, with i initially set to 2) meets the preset conditions (the turning angle is greater than the turning angle threshold and greater than the turning angle of the previous sub-time period). If so, we increment the number of turns by 1 (initially set to 0); otherwise, we increment the number of no turns by 1 (initially set to 0). We then check whether the i-th turning angle is the last one. If it is the last one, we start calculating the confidence score. If it is not the last one, we increment the value of i by 1 and continue with this step.

[0281] Finally, after the previous step, the number of turns and the number of non-turns within a time period A can be obtained. It should be noted that this time period A (e.g., 3 seconds, which can be denoted as the first time period) can be the same as a time window a (which can be denoted as the first time window), or it can be longer than a time window a. For example, if a time window a is 1 second and the time period A is 3 seconds, the center moments of the time period A and time window a can coincide. This is equivalent to using statistical data longer than a time window as the statistical result for this time window. Of course, the focus of this embodiment is not on the division of individual time points; therefore, in practical applications, the number of turns and the number of non-turns within a time window can also be obtained using other time measurement methods. Based on the number of turns and the number of non-turns within a time period A, the confidence level of time window a is obtained.

[0282] In practical applications, the confidence level within the current time window is calculated based on the following formula.

[0283]

[0284] Where CI-2 represents the confidence level, turn_count is the number of turns (if turn_count is greater than 10, then turn_count is 10), noTurn is the number of no turns, t and a are both penalty factors, t is 20 and a is 50.

[0285] As mentioned earlier, the confidence level of the scene obtained by the geomagnetic prediction module can also be obtained. For details, please refer to... Figure 9 The flowchart shown illustrates the process of calculating confidence levels in the geomagnetic prediction module.

[0286] Reference Figure 10 As shown, firstly, data collected by the accelerometer and gyroscope within a certain period (e.g., a time window) are acquired to obtain the number of steps in each sub-time period within that period; then, the data sequence collected by the geomagnetic sensor within a certain period (e.g., a time window) is acquired, and the geomagnetic data sequence is smoothed to obtain the smoothed geomagnetic data sequence.

[0287] Based on the number of steps in each sub-time period, it is determined whether a peak in the geomagnetic data sequence is a primary peak. In practical applications, peaks in geomagnetic data sequences may be fluctuations caused by acquisition errors (unrelated to motion) or fluctuations caused by movement (related to motion). Therefore, the number of primary peaks caused by movement within a time window can be determined based on the number of steps in each sub-time period.

[0288] Finally, the confidence level is obtained based on the number of main peaks.

[0289] The confidence level is calculated as follows:

[0290]

[0291]

[0292] Where CI-3 represents the confidence level of the geomagnetic prediction module for the given time window, PeakNum is the number of main peaks within the given time window, Maxsample is the number of samples within the given time window, K is the sliding window size, and thre represents the threshold for the number of peaks.

[0293] As mentioned earlier, the first sub-prediction result (scene and first confidence level) has been obtained based on the XGBoost model, and the second sub-prediction result (scene and second confidence level) and the third sub-prediction result (scene and third confidence level) have been obtained based on the geomagnetic prediction module and the turning prediction module, respectively.

[0294] The three sub-prediction results can be fused using a Bayesian voting strategy to obtain a common prediction result. For example, this could be applied to scenarios corresponding to different time windows. Of course, in practical applications, one or more sub-prediction results can also be selected for fusion.

[0295] In practical implementation, when the data is in motion and there is a large amount of data, the geomagnetic prediction module and the turning prediction module can be selected for use.

[0296] As an example, in Figure 8 In the five scenarios shown, in the third scenario, the second and third sub-prediction results are obtained using the geomagnetic prediction module and the turning prediction module, respectively. Then, the prediction results from XGBoost, the geomagnetic prediction module, and the turning prediction module, along with their corresponding weights, are fused to obtain the fused prediction result. Alternatively, in the third scenario, only the sub-prediction results from the geomagnetic prediction module or the turning prediction module can be fused to obtain the fused prediction result.

[0297] In the other four cases, the prediction is performed without using the geomagnetic prediction module and the turning prediction module, or the prediction is performed using the geomagnetic prediction module and the turning prediction module, and then the prediction results of the geomagnetic prediction module and the turning prediction module are not used. That is, the fused prediction result is the first sub-prediction result of the XGBoost module.

[0298] Of course, a user's walking trajectory may contain sub-trajectories corresponding to one or more of the above five scenarios. For each sub-trajectory, the corresponding sub-model in the XGBoost model is selected to obtain the first sub-prediction result; for each sub-trajectory, it is then determined whether to use the prediction results from the geomagnetic prediction module and the turning prediction module for fusion.

[0299] After obtaining the prediction result after fusing the three (or possibly one or two in practice), it is equivalent to obtaining the indoor and outdoor scenes corresponding to each time window in the user's walking trajectory. The boundary between the outdoor and indoor scenes is the switching point between the indoor and outdoor scenes. In some cases, the switching point between the indoor and outdoor scenes can also be adjusted.

[0300] As mentioned earlier, changes in the number of satellites monitored by the GNSS module on electronic devices are related to buildings.

[0301] When a user is carrying an electronic device in a building with small entrances and exits, such as a residential building, the number of satellites detected by the GNSS module on the electronic device increases rapidly at the exit when the user moves from indoors to outdoors; similarly, the number of satellites detected by the GNSS module on the electronic device decreases rapidly at the entrance when the user moves from outdoors to indoors. Typically, in this embodiment, the entrance and exit are the same location; however, moving from indoors to outdoors is referred to as the exit, and moving from outdoors to indoors is referred to as the entrance.

[0302] Reference Figure 11 As shown in (a), when a user is carrying an electronic device in a building with a large entrance, such as a shopping mall or office building, if the user is carrying the electronic device from indoors to outdoors, the number of satellites detected by the GNSS module on the electronic device will start to increase when the user is still some distance away from the exit.

[0303] Similarly, refer to Figure 11 As shown in (b), when a user is carrying an electronic device in a building with a large entrance, such as a shopping mall or office building, the number of satellites detected by the GNSS module on the electronic device only begins to decrease after the user has traveled a certain distance from the entrance.

[0304] Figure 11 The number of satellites shown in the diagram is only used to compare outdoor, shallow indoor, and deep indoor conditions, and does not represent an absolute number.

[0305] Based on the above analysis, it can be understood that the satellite change points for buildings with large entrances, such as shopping malls and office buildings, are closer to the interior of the building compared to buildings with smaller entrances, such as residential buildings. Therefore, for walking trajectories from outdoors to indoors, if the original prediction points the switching point are in the middle or after the satellite has fully descended, the switching point needs to be corrected to the beginning of the descent. This way, regardless of the size of the building entrance, entrance latency can be reduced, and the response speed for switching between indoor and outdoor environments can be improved.

[0306] Therefore, the correction module can correct the entry point based on the number of satellites detected by the GNSS sensor and the above prediction results.

[0307] Before making any specific corrections, if the entry point is located in the middle stage of the descent, the following situations may occur:

[0308] Reference Figure 12As shown in (a), if the predicted entry point occurs in the middle of the satellite count decline phase, and the number of satellites stabilizes at N2 (e.g., greater than 5 satellites) before the decline phase and at N1 (e.g., less than 5 satellites) after the decline phase, but the delay between the predicted entry point and the start of the satellite count decline phase is short (e.g., less than 5 samples, 2.5 time windows), then no correction is needed for the entry point. Because of this, the accuracy of the entry point obtained from multiple sensors may be better than the accuracy of the satellite count obtained from a GNSS sensor.

[0309] Reference Figure 12 As shown in (b), if the predicted entry point occurs in the middle of the satellite count decline phase, with the satellite count stabilizing at N2 (e.g., greater than 5 satellites) before the decline phase and stabilizing at N1 (e.g., less than 5 satellites) after the decline phase, and if there is a significant delay between the predicted entry point and the start of the satellite count decline phase (e.g., greater than or equal to 5 samples, 2.5 time windows), then the entry point needs to be corrected. Because in this case, entry points obtained from multiple sensors may have discrepancies, and for consistency, the entry point can be corrected to the start of the satellite count decline phase.

[0310] Reference Figure 12 As shown in (c), the predicted entry point is in the middle stage of the satellite count decrease, and the number of satellites after the decrease stabilizes at N1 (e.g., greater than 5 satellites). Therefore, the number of satellites before the decrease stage stabilizes at N2 (N2 must be greater than 5 satellites), indicating that the satellite count decrease phase is entirely outdoors. For example, if a user walks from an open area to under trees or from an open area to an area with many buildings, although there is indeed a decrease in the number of satellites, the number of satellites after the decrease is still relatively high. In this case, the user is still actually outdoors, and the indoor scene after the entry point in the decrease phase needs to be corrected to an outdoor scene.

[0311] Reference Figure 12 As shown in (d), the predicted entry point is in the middle stage of the satellite count decrease, and the number of satellites before the decrease is stable at N1 (e.g., less than 5 satellites). Then, the number of satellites after the decrease stage is stable at N2 (N2 must be less than 5 satellites), indicating that the decrease in the number of satellites occurs entirely indoors. For example, when a user walks from a shallow area to a deeper area in an indoor scene, although there is indeed a decrease in the number of satellites, the number of satellites is still relatively small even before the decrease. Therefore, the outdoor scene before the entry point in this decrease stage needs to be corrected to an indoor scene.

[0312] In this way, the entrance in a walking trajectory can be corrected to a suitable position, while situations that are not entrances can be corrected.

[0313] In practical implementation, it can be done according to Figure 13 The flowchart shown is used to modify the entry point.

[0314] Reference Figure 13 As shown, the number of satellites detected by the GNSS sensor and the prediction results before correction (outdoor or indoor scenes corresponding to each time window) are obtained. The prediction results corresponding to each time window in this stage are used as a sample.

[0315] Then, the number of satellites is filtered to obtain the filtered number of satellites. Due to the limitation of the sampling frequency of the data acquisition unit, the number of satellites cannot be updated in a timely manner, and there may even be cases where a certain sample is not updated. Therefore, it is necessary to perform filtering and smoothing processing on the number of satellites.

[0316] Finally, it is determined whether each sample (the prediction result of the i-th time window, with i initially set to 2) is a predicted entry point (e.g., the i-th time window is for an indoor scene, and the (i-1)-th time window is for an outdoor scene). If it is a predicted entry point, the predicted entry point is corrected; if a correction is made (satisfying the following conditions is met), the prediction is corrected. Figure 12 If the correction shown in (b) is corrected, then it is determined whether the current i-th sample is the last sample. If yes, the process ends; otherwise, the above processing continues for the (i+1)-th sample. Of course, if no correction is performed at the entry point of the i-th sample (satisfying the condition...), then... Figure 12 If the cases are (a), (c), or (d), then process them accordingly; after processing, determine whether the current i-th sample is the last sample. If yes, end; otherwise, continue the above processing for the (i+1)-th sample.

[0317] Of course, the correction process can be performed cyclically, processing each sample one by one in chronological order. Therefore, while satisfying... Figure 12 In the case shown in (c), only the indoor scene corresponding to the current time window can be switched to the outdoor scene, because it will still be judged as the entry point in the next sample, and will still satisfy the condition. Figure 12 As shown in (c)... if the loop continues, it will keep correcting until it reaches a sample that does not belong to the entry point (the previous sample was corrected to be outdoors, and this sample itself is outdoors).

[0318] The time-related parameter values ​​and time-related division methods in the above embodiments are for illustrative purposes only and can be adjusted in actual applications.

[0319] The indoor / outdoor scene recognition method provided in this application can identify whether each point belongs to an indoor or outdoor scene, and can also obtain the transition points between indoor and outdoor. After extensive testing and statistics, the accuracy statistics of the indoor / outdoor scene recognition method provided in this application are shown in Table 9.

[0320] Table 9. Accuracy of Indoor and Outdoor Scene Recognition

[0321] Modes Accuracy indoor 97.17％ outdoor 94.82％ average 96％ Overall accuracy 96％

[0322] The indoor / outdoor scene recognition method provided in this application embodiment can also identify the switching points between indoor and outdoor areas. Therefore, it can also target the average delay of entrances and exits; the proportion of entrances and exits with a predicted offset of less than or equal to 14 samples (approximately 7 seconds with a time window of 1 second and 50% data overlap); and the proportion of entrances and exits with a predicted offset of less than or equal to 20 samples (approximately 10 seconds) in the correctly predicted trajectory. See Table 10 for details.

[0323] Table 10 Parameters Related to Entrance / Exit Delay

[0324]

[0325] As another embodiment of this application, the indoor and outdoor scene recognition method provided in this application includes:

[0326] Acquire data corresponding to the user's walking trajectory, wherein the data corresponding to the user's walking trajectory is time-related data collected by multiple sensors on the electronic device carried by the user while walking;

[0327] Based on the data corresponding to the user's walking trajectory, determine the target prediction module used to identify indoor and outdoor scenes;

[0328] The prediction result is obtained based on the data related to the target prediction module in the data corresponding to the walking trajectory;

[0329] Based on the satellite data collected by the satellite navigation system in the data corresponding to the walking trajectory, the prediction result is corrected to obtain the recognition result. The recognition result includes the scene corresponding to each time window in the walking trajectory, and the scene includes indoor scene or outdoor scene.

[0330] In this embodiment, a target prediction module for identifying indoor and outdoor scenes can be determined based on data collected by multiple sensors on an electronic device carried by the user while walking. Then, the corresponding data is input into the corresponding target prediction module to obtain the prediction result of the indoor and outdoor scenes. After obtaining the prediction result, the prediction result can be corrected based on satellite data collected by the satellite navigation system to obtain the recognition result of the indoor and outdoor scenes. This method of determining different prediction modules under different conditions can balance prediction efficiency and prediction accuracy. At the same time, the method of correcting the prediction result with satellite data can improve the prediction accuracy.

[0331] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0332] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0333] This application also provides a computer program product that, when run on a first device, enables the first device to implement the steps described in the various method embodiments above.

[0334] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0335] This application also provides a chip system, which includes a processor coupled to a memory. The processor executes a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0336] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0337] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0338] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for recognizing indoor and outdoor scenes, characterized in that, include: Acquire data corresponding to the user's walking trajectory, wherein the data corresponding to the user's walking trajectory is time-related data collected by multiple sensors on the electronic device carried by the user while walking; Based on the data corresponding to the user's walking trajectory, determine the target prediction module used to identify indoor and outdoor scenes; The prediction result is obtained based on the data related to the target prediction module in the data corresponding to the walking trajectory; Based on the satellite data collected by the satellite navigation system in the data corresponding to the walking trajectory, the prediction result is corrected to obtain the recognition result. The recognition result includes the scene corresponding to each time window in the walking trajectory, and the scene includes indoor scene or outdoor scene. The target prediction module for identifying indoor and outdoor scenes based on data corresponding to the user's walking trajectory includes: The user's state is determined based on the data corresponding to the user's walking trajectory. The user's state includes a moving state or a non-moving state. Based on the user's status and the number of samples in the data corresponding to the walking trajectory, a target prediction module for identifying indoor and outdoor scenes is determined. The target prediction module is at least one of a plurality of pre-set prediction modules.

2. The identification method as described in claim 1, characterized in that, The step of determining the target prediction module used for identifying indoor and outdoor scenes based on the user's state and the number of samples in the data corresponding to the walking trajectory includes: If the user is in motion and the number of samples in the data corresponding to the walking trajectory is less than a first value, then the target prediction module used to identify indoor and outdoor scenes includes: a decision tree classifier, and the sub-target prediction module used in the decision tree classifier includes a joint prediction module, which is used to predict data collected by multiple sensors. If the user is in motion, and the number of samples in the data corresponding to the walking trajectory is not less than the first value and less than the second value, then the target prediction module used to identify indoor and outdoor scenes includes: the decision tree classifier, and the sub-target prediction module used in the decision tree classifier includes the ambient light prediction module and the joint prediction module. If the user's state is in motion and the number of samples in the data corresponding to the walking trajectory is not less than the second value, then the target prediction module used to identify indoor and outdoor scenes includes at least one of a turning prediction module and a geomagnetic prediction module, as well as the decision tree classifier. The sub-target prediction module used in the decision tree classifier includes at least one of an ambient light prediction module, a satellite prediction module, and a base station prediction module, as well as the joint prediction module. If the user is in a non-moving state and the number of samples in the data corresponding to the walking trajectory is less than the third value, then the target prediction module used to identify indoor and outdoor scenes includes: the decision tree classifier, the sub-target prediction module used in the decision tree classifier includes the joint prediction module, and the third value is greater than the first value; If the user is in a non-moving state and the number of samples in the data corresponding to the walking trajectory is not less than the third value, then the target prediction module used to identify indoor and outdoor scenes includes: the decision tree classifier, and the sub-target prediction module used in the decision tree classifier includes the ambient light prediction module and the joint prediction module.

3. The method as described in claim 2, characterized in that, Determining the user's status based on the data corresponding to the user's walking trajectory includes: Based on the data collected by the accelerometer and gyroscope sensors in the data corresponding to the user's walking trajectory, the number of steps taken by the user in the user's walking trajectory is determined. The user's status is determined based on the number of steps the user takes in the user's walking trajectory.

4. The identification method as described in claim 2, characterized in that, The input data for the joint prediction module includes data collected from at least two of the following: the data corresponding to the walking trajectory, from the geomagnetic sensor, the accelerometer, the gyroscope sensor, the satellite navigation module, the wireless communication module, and the mobile communication module.

5. The identification method as described in claim 2, characterized in that, The step of obtaining a prediction result based on data related to the target prediction module from the data corresponding to the walking trajectory includes: Based on the sub-target prediction module in the decision tree classifier and the input data corresponding to the sub-target prediction module, the first sub-prediction result corresponding to the decision tree classifier is obtained; When the target prediction module includes the turning prediction module, the second sub-prediction result corresponding to the turning prediction module is obtained based on the turning prediction module and the input data corresponding to the turning prediction module. When the target prediction module includes the geomagnetic prediction module, the third sub-prediction result corresponding to the geomagnetic prediction module is obtained based on the geomagnetic prediction module and the input data corresponding to the geomagnetic prediction module. The prediction result is obtained by fusing the sub-prediction results corresponding to each target prediction module.

6. The identification method as described in claim 5, characterized in that, The step of obtaining the first sub-prediction result corresponding to the decision tree classifier based on the sub-target prediction module in the decision tree classifier and the input data corresponding to the sub-target prediction module includes: Based on the prediction outputs and corresponding weights of each sub-target prediction module in the decision tree classifier, the first sub-prediction result corresponding to the decision tree classifier is obtained.

7. The identification method as described in claim 5, characterized in that, The prediction result is obtained by fusing the sub-prediction results corresponding to each target prediction module, including: The prediction result is obtained based on the sub-prediction result corresponding to each target prediction module and the weight corresponding to each target prediction module. The sub-prediction result includes the predicted scenario and confidence level, and the weight corresponding to the target prediction module is related to the confidence level in the sub-prediction result corresponding to the target prediction module.

8. The method according to any one of claims 5 to 7, characterized in that, The calculation method for the second confidence level in the second sub-prediction result includes: Based on the data collected by the accelerometer and the gyroscope in the data corresponding to the walking trajectory, the number of turns and the number of non-turns are obtained for each time window in the walking trajectory. Based on the number of turns and the number of non-turns corresponding to each time window in the walking trajectory, the second confidence level corresponding to each time window in the walking trajectory is obtained.

9. The method as described in claim 8, characterized in that, The data collected by the accelerometer and gyroscope sensors in the data corresponding to the walking trajectory are used to obtain the number of turns and the number of non-turns for each time window in the walking trajectory, including: Calculate the average acceleration value of the acceleration collected by the accelerometer in each sub-time period of the first time period, wherein the first time period includes a first time window, and the first time window is any time window in the walking trajectory, and the duration of each sub-time period is less than the duration of the first time window. Calculate the average angular velocity of the angular velocities collected by the gyroscope sensor in each sub-time period within the first time period; Calculate the projection of the mean angular velocity in each sub-time period of the first time period onto the direction of the mean acceleration to obtain the turning angle corresponding to each sub-time period; The number of sub-time periods in which the turning angle is greater than the angle threshold and the turning angle is greater than the turning angle of the previous sub-time period is taken as the number of turns corresponding to the first time window. The number of sub-time periods in which the turning angle is not greater than the angle threshold, or the turning angle is not greater than the turning angle of the previous sub-time period, is taken as the number of times the first time window does not turn.

10. The method as described in claim 9, characterized in that, The center time of the first time period is the same as the center time of the first time window, and the duration of the first time period is greater than or equal to the duration of the first time window.

11. The method as described in claim 8, characterized in that, The step of obtaining the second confidence level corresponding to each time window of the walking trajectory based on the number of turns and the number of non-turns corresponding to each time window includes: according to Calculate the second confidence level for each time window in the walking trajectory; Where CI-2 represents the second confidence level, turn_count is the number of turns (if turn_count is greater than 10, then turn_count takes the value 10), noTurn is the number of no turns, and t and a are both penalty factors.

12. The method as described in claim 5, characterized in that, The calculation method for the third confidence level in the third sub-prediction result includes: Based on the data collected by the accelerometer and gyroscope sensors in the data corresponding to the walking trajectory, the number of walking steps corresponding to each time window in the walking trajectory is obtained; Based on the data collected by the geomagnetic sensor in the data corresponding to the walking trajectory and the number of walking steps, the number of main peaks of the geomagnetic data corresponding to each time window in the walking trajectory is obtained. The third confidence level is obtained based on the number of main peaks of the geomagnetic data corresponding to each time window in the walking trajectory.

13. The method as described in claim 12, characterized in that, The step of obtaining the third confidence level corresponding to each time window of the walking trajectory based on the number of main peaks of the geomagnetic data corresponding to each time window includes: according to Calculate the third confidence level for each time window in the walking trajectory; Where CI-3 represents the third confidence level, PeakNum represents the number of main peaks in the geomagnetic data within the time window, Maxsample represents the number of geomagnetic samples within the time window, K represents the size of the time window, and thre represents the threshold for the number of peaks.

14. The method according to any one of claims 1 to 7, characterized in that, The prediction result is corrected based on satellite data collected by the satellite navigation system in the data corresponding to the walking trajectory to obtain the identification result, including: The prediction results of each time window in the walking trajectory are viewed in turn. If the prediction result of the previous time window is an outdoor scene and the prediction result of the current time window is an indoor scene, it is determined whether the current time window is in the stage of decreasing satellite number. If the current time window is in the middle of the decreasing phase of the number of satellites, the number of satellites before the decreasing phase of the current time window is greater than the first number, the number of satellites after the decreasing phase of the current time window is less than the second number, and the event interval between the current time window and the starting position of the decreasing phase of the current time window is less than the first duration, then no correction is made, and the first number is greater than or equal to the second number. If the current time window is in the middle of a phase in which the number of satellites decreases, the number of satellites before the current time window's phase of decrease is greater than a first number, the number of satellites after the current time window's phase of decrease is less than a second number, and the event interval between the current time window and the starting position of the current phase of decrease is not less than a first duration, then the entry point is corrected to the starting position of the current phase of decrease, and the scene in the time window between the starting position of the current phase of decrease and the current time window is corrected to an indoor scene. If the current time window is in the middle of the decreasing phase of the number of satellites, and the number of satellites before the decreasing phase of the current time window is greater than the first number, and the number of satellites after the decreasing phase of the current time window is not less than the second number, then the scene of the current time window will be corrected to an outdoor scene. If the current time window is in the middle of a phase where the number of satellites is decreasing, and the number of satellites before the current time window is decreasing is no greater than the first number, and the number of satellites after the current time window is decreasing is greater than the second number, then the scene in the time window before the current time window will be corrected to an indoor scene, until the scene in a certain time window is corrected to an indoor scene.

15. An electronic device, characterized in that, The electronic device includes a first processor and a second processor, the first processor and the second processor being configured to run a computer program stored in a memory, such that the electronic device implements the method as described in any one of claims 1 to 14.

16. A chip system, characterized in that, The method includes a first processor and a second processor, the first processor being coupled to a memory, the first processor executing a computer program stored in the memory to implement the method as described in any one of claims 1 to 14.

Images