Scene adaptive information processing method and device, equipment and storage medium

Abstract

The invention discloses a scene self-adaptive information processing method and device, equipment and a storage medium, and relates to the technical field of data processing, and the method comprises the steps: carrying out the preprocessing of multi-source sensing data, and obtaining a fusion observation sequence; the multi-source sensing data comprises GPS positioning data, distance sensor data, motion sensor data and physiological sensor data; inputting the fused observation sequence into a scene recognition model to obtain a current scene identifier; and generating multi-modal feedback information based on the current scene identifier. According to the invention, information processing is carried out by fusing multi-source sensing data, the problem that a single sensor is susceptible to environmental interference is eliminated, and the environmental and user state judgment precision is improved. Meanwhile, the current scene of the user is accurately obtained by means of the scene recognition model, the multi-modal feedback information is made to adapt to the scene requirement, the accuracy and usability of equipment assistance are improved, and the operation burden of the user with visual impairment is reduced.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for scene adaptive information processing. Background Technology

[0002] The algorithms of existing assistive devices for visually impaired users are mostly designed for single functions (such as navigation or voice interaction only), which have problems such as high response latency and low efficiency of multi-task collaboration, resulting in low assistive efficiency and poor practicality for visually impaired people.

[0003] Therefore, how to improve the accuracy and ease of use of assistive devices and effectively reduce the operational burden on visually impaired people has become an urgent problem to be solved. Summary of the Invention

[0004] The main objective of this application is to provide a scene-adaptive information processing method, apparatus, device, and storage medium, aiming to solve the technical problem of how to improve the accuracy and ease of use of assistive devices and effectively reduce the operational burden on visually impaired people.

[0005] To achieve the above objectives, this application proposes a scene adaptive information processing method, which includes: The multi-source sensor data is preprocessed to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data. The fused observation sequence is input into the scene recognition model to obtain the current scene identifier; Multimodal feedback information is generated based on the current scene identifier.

[0006] In one embodiment, the step of preprocessing multi-source sensor data to obtain a fused observation sequence includes: Multi-source sensor data is subjected to pre-defined standardization processing to obtain standard sensor data; Perform spatiotemporal alignment preprocessing on the standard sensor data to obtain standardized subsequences; The standardized subsequences are weighted and fused based on the real-time confidence scores of the sensors to obtain a fused observation sequence.

[0007] In one embodiment, the step of generating multimodal feedback information based on the current scene identifier includes: Based on the current scene identifier, determine the set of real-time function strategies corresponding to the preset function modules; Multimodal feedback information is generated through the set of real-time functional strategies.

[0008] In one embodiment, the preset functional modules include: a navigation module, an obstacle warning module, a Bluetooth positioning module, a scene description module, a physiological monitoring module, and a gait correction module; The step of determining the set of real-time function strategies corresponding to the preset function modules based on the current scene identifier includes: The current scene identifier is converted into the current weight vector according to the preset strategy mapping table; Based on the current weight vector, the functions of the navigation module, the obstacle warning module, the Bluetooth positioning module, the scene description module, the physiological monitoring module, and the gait correction module are prioritized to form a real-time function strategy set.

[0009] In one embodiment, the step of generating multimodal feedback information through the real-time function strategy set includes: Obtain the early warning level corresponding to the fused observation sequence; Multimodal feedback information is generated based on the warning level and the set of real-time functional strategies.

[0010] In one embodiment, the step of generating multimodal feedback information based on the warning level and the real-time functional strategy set includes: Obtain personalized configuration data and historical user behavior data; Based on the historical user behavior data, the monitored user interaction operations are optimized and identified to obtain the user's operation intent; Multimodal feedback information is generated based on the personalized configuration data, the user's operation intent, the warning level, and the set of real-time functional strategies.

[0011] In one embodiment, after preprocessing the multi-source sensor data to obtain the fused observation sequence, the method further includes: When a preset abnormal signal is detected in the fused observation sequence, the abnormal event identifier corresponding to the preset abnormal signal is obtained; Early warning response information is generated based on the abnormal event identifier.

[0012] Furthermore, to achieve the above objectives, this application also proposes a scene-adaptive information processing device, which includes: The data processing module is used to preprocess multi-source sensor data to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data. The scene recognition module is used to input the fused observation sequence into the scene recognition model to obtain the current scene identifier; An adaptive feedback module is used to generate multimodal feedback information based on the current scene identifier.

[0013] In addition, to achieve the above objectives, this application also proposes a scene adaptive information processing device, which includes: a memory, a processor, and a scene adaptive information processing program stored in the memory and executable on the processor. The scene adaptive information processing program is configured to implement the steps of the scene adaptive information processing method described above.

[0014] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, storing a program that implements the scene adaptive information processing method. The program that implements the scene adaptive information processing method is executed by a processor to implement the steps of the scene adaptive information processing method as described above.

[0015] This application provides a scene-adaptive information processing method, apparatus, device, and storage medium. The method includes: preprocessing multi-source sensor data to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data; inputting the fused observation sequence into a scene recognition model to obtain a current scene identifier; and generating multimodal feedback information based on the current scene identifier. This application first eliminates the problem of single-sensor susceptibility to environmental interference by fusing multi-source sensor data for information processing, improving the accuracy of environmental and user state judgment. Simultaneously, this application accurately obtains the user's current scene using a scene recognition model, allowing the multimodal feedback information to adapt to scene requirements, improving the accuracy and ease of use of the device, and effectively reducing the operational burden on visually impaired users. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a first flowchart illustrating the first embodiment of the adaptive information processing method for the scenario described in this application. Figure 2 This is a second flowchart illustrating the first embodiment of the adaptive information processing method for the scenario described in this application. Figure 3 This is a schematic diagram of the first process of the second embodiment of the adaptive information processing method for the scenario of this application; Figure 4This is a schematic diagram of the second process of the second embodiment of the adaptive information processing method for the scenario of this application; Figure 5 This is a schematic diagram of the third process of the second embodiment of the adaptive information processing method for the scenario of this application; Figure 6 This is a schematic diagram of the module structure of the scene adaptive information processing device in the embodiments of this application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the scene adaptive information processing method in the embodiments of this application.

[0019] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The main solution of this application is: preprocessing multi-source sensor data to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data; inputting the fused observation sequence into a scene recognition model to obtain the current scene identifier; and generating multimodal feedback information based on the current scene identifier.

[0023] Existing assistive devices for visually impaired users often employ algorithms designed for single functions (such as navigation or voice interaction only), resulting in issues like high response latency and low efficiency in multi-task collaboration. For example, traditional navigation algorithms do not consider the reliance of visually impaired users on tactile feedback, obstacle warning algorithms are susceptible to environmental interference leading to false alarms, and voice interaction algorithms have low accuracy in recognizing commands with accents. Consequently, these devices offer low assistive efficiency and poor practicality for visually impaired individuals.

[0024] To address the issues of fragmented functionality, weak scene adaptability, and low interaction efficiency in existing algorithms, this application proposes a method for providing precise assistance to visually impaired individuals by dynamically adjusting information processing strategies and feedback methods based on the user's real-time scenario. The core of this method is multi-source data fusion and scene recognition, enabling the device's functions to adapt to different usage scenarios. Therefore, this application eliminates the problem of single-sensor susceptibility to interference (such as inaccurate GPS indoor positioning and large detection errors of distance sensors under strong light) by fusing multi-source sensor data for information processing, improving the accuracy of environmental and user status judgment. Furthermore, it utilizes a scene recognition model to accurately acquire the current scene, allowing multimodal feedback information to adapt to scene requirements, reducing the operational burden on visually impaired users, and improving the accuracy and ease of use of the device.

[0025] It should be noted that the execution subject in this embodiment can be a scene-adaptive information processing system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, smartwatch, etc., or a scene-adaptive information processing device capable of realizing the above functions, etc. This embodiment does not specifically limit it. The following uses a scene-adaptive information processing device (hereinafter referred to as processing device) as the execution subject as an example to describe this embodiment and the following embodiments.

[0026] Based on this, embodiments of this application provide a scene adaptive information processing method, referring to... Figure 1 , Figure 1 This is a first flowchart illustrating the first embodiment of the adaptive information processing method for the scenario described in this application.

[0027] In this embodiment, the scene adaptive information processing method includes steps S10 to S30: Step S10: Preprocess the multi-source sensor data to obtain the fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data. It is understood that the aforementioned multi-source sensor data can be real-time data collected by various sensors mounted in the processing device at a preset frequency (e.g., 100ms / time). This data may include GPS positioning data for obtaining the user's geographical location and outdoor scene positioning basis; distance sensor data for detecting the distance to obstacles around the user; motion sensor data for capturing the user's gait, movement speed, and direction, which can be obtained from accelerometers and gyroscopes (or inertial measurement units); and physiological sensor data for monitoring the user's health status, which may include heart rate sensor and blood oxygen sensor data.

[0028] At this point, the processing equipment can further perform standardization, noise reduction, and outlier removal on the multi-source sensor data, i.e., the above preprocessing, to eliminate data format differences and noise interference, and provide high-quality data for subsequent fusion.

[0029] In one feasible implementation, refer to Figure 2 , Figure 2 This is a second flowchart illustrating the first embodiment of the adaptive information processing method for this application scenario. In this embodiment, step S10 may include steps A1 to A3: Step A1: Perform pre-defined standardization processing on the multi-source sensor data to obtain standard sensor data; In essence, during the pre-defined standardization process, the processing equipment can uniformly process multi-source sensor data according to pre-set rules, including data format standardization, data range standardization, and data noise reduction, ensuring data quality and consistency. At this point, the standard sensor data after pre-defined standardization is sensor data with unified format, noise elimination, and outlier removal, which can be directly used for subsequent spatiotemporal alignment and weighted fusion.

[0030] Furthermore, the processing device can perform targeted data processing on different sensor data, while unifying the data format and timestamp of all sensor data. For example, the processing device can perform error correction on GPS positioning data. For instance, the processing device can first convert the raw latitude and longitude data (such as "113°55′22.8″E, 22°32′42″N") into decimal format ("113.923°E, 22.545°N"), then correct the error using differential positioning technology (such as correcting the original error of 5 meters to within 1 meter), and finally record it in the format of "timestamp, GPS positioning, longitude, latitude".

[0031] Then, the distance sensor data is filtered and denoised. For example, the Kalman filter algorithm is used to remove noise caused by strong light and dust (such as smoothing the original fluctuation range of 0.5-3 meters to 1-2.8 meters). The data is then recorded in the format of "timestamp, distance detection, relative distance to obstacle, obstacle direction" (such as "1690000000100, distance detection, 2 meters, directly in front").

[0032] Simultaneously, gait features are extracted from motion sensor data. For example, gait features (gait cycle 0.8 - 1.2 seconds, stride length 0.6 - 0.8 meters) are extracted using a sliding window algorithm (window size 500ms), abnormal gait data (such as erroneous data where stride length suddenly changes to 1.5 meters) are removed, and the data is recorded in the format of "timestamp, motion feature, gait cycle, stride length, motion direction".

[0033] Finally, outlier removal is performed on the physiological sensor data. For example, the 3σ criterion is used to remove outliers (such as erroneous data such as a sudden increase in heart rate from 80 beats / minute to 180 beats / minute). The heart rate and blood oxygen data are standardized into the format of "timestamp, physiological index, heart rate, blood oxygen saturation" (such as "1690000000100, physiological index, 82 beats / minute, 98%").

[0034] Step A2: Perform spatiotemporal alignment preprocessing on the standard sensor data to obtain a standardized subsequence; It's important to understand that the processing device can perform time synchronization and spatial coordinate unification on standard sensor data, i.e., the aforementioned spatiotemporal alignment preprocessing, to facilitate subsequent weighted fusion. During time synchronization, the processing device's built-in high-precision clock can be used as a reference, and timestamp calibration can control the timestamp error of each sensor's data to ≤10ms, ensuring data correspondence at the same time point. In the spatial coordinate unification process, coordinate system mapping transformation can be used. For example, the relative distance to an obstacle detected by the distance sensor (e.g., "2 meters ahead") can be converted to absolute coordinates consistent with GPS (e.g., "113.923°E, 22.545°N 2 meters ahead, corresponding to coordinates 113.92302°E, 22.545°N"), ensuring data fusion across the spatiotemporal dimensions.

[0035] At this point, the continuous data segments composed of standard sensor data after spatiotemporal alignment preprocessing, arranged in chronological order, constitute the aforementioned standardized subsequence. For example, a set of data is collected every 100ms, and 10 consecutive sets of data form a standardized subsequence (time span of 1 second). The data within the subsequence have consistent spatiotemporal dimensions and can be directly used for weighted fusion.

[0036] Step A3: Based on the real-time confidence of the sensors, the standardized subsequences are weighted and fused to obtain the fused observation sequence.

[0037] It is easy to understand that the above-mentioned fused observation sequence can be a continuous data sequence formed by integrating preprocessed standardized subsequences through a weighted fusion algorithm. It can comprehensively reflect the user's environmental state (location, obstacle distribution) and the user's own state (gait, physiological indicators). The data update frequency can be once every 100ms.

[0038] The aforementioned real-time confidence level of the sensor can be a quantitative indicator reflecting the reliability of the sensor's data in the current scenario (it can take a value from 0 to 1, with a higher value indicating a higher probability). It can be obtained by multiplying the sensor performance benchmark value built into the processing device by the scene correction coefficient. The sensor performance benchmark value is preset in the processing device based on the sensor's factory parameters and long-term testing. The scene correction coefficient can be set according to the degree of influence of each scenario on the sensor (e.g., a correction coefficient of 0.7 for distance sensors in indoor environments and 0.9 for outdoor environments), and can be determined based on the recognition results of the user's environment scene in real time. Understandably, the scene correction coefficient is usually at a default value at initial startup, but can be updated in real time based on the real-time scene recognition results during subsequent processes.

[0039] For example, taking an indoor scene as an example, the GPS performance baseline value is 0.3, and the indoor scene correction coefficient is 1 (the indoor scene correction for GPS is usually included in the performance baseline value), then the real-time confidence of GPS is 0.3; the distance sensor performance baseline value is 0.85, and the indoor scene correction coefficient is 0.7 (the indoor lighting is relatively dim, so the impact on the distance sensor is small), then the real-time confidence of the distance sensor is 0.85 × 0.7 = 0.595; the motion sensor performance baseline value is 0.8, and the indoor scene correction coefficient is 0.9 (the indoor motion state is stable, so the impact on the motion sensor is small), then the real-time confidence of the motion sensor is 0.8 × 0.9 = 0.72; the physiological sensor performance baseline value is 0.9, and the indoor scene correction coefficient is 0.95 (the indoor environment is stable, so the impact on the physiological sensor is not significant), then the real-time confidence of the physiological sensor is 0.9 × 0.95 = 0.855.

[0040] At this point, after determining the real-time confidence level corresponding to each sensor data, the processing device can perform weighted Kalman filtering on each sensor data in the standardized subsequence based on the real-time confidence level to generate the fused environmental state (position accuracy ±1 meter, obstacle distance accuracy ±0.3 meters) and user state (gait type, movement speed, physiological indicators), which is the aforementioned fused observation sequence.

[0041] Step S20: Input the fused observation sequence into the scene recognition model to obtain the current scene identifier; It should be noted that the above scene recognition model can be a model pre-built based on deep learning technology. It can adopt a fusion architecture of CNN (Convolutional Neural Network) and LSTM (Long Short-Term Memory Network) and be trained with more than 100,000 labeled scene samples. This scene recognition model can recognize indoor (home / shopping mall), outdoor (street / park), transportation (bus / subway), sports and other scenes, and update the scene judgment result every 500ms.

[0042] Accordingly, the current scene identifier can be the scene category result output by the scene recognition model based on the fused observation sequence, such as indoor (home / shopping mall), outdoor (street / park), transportation (bus / subway), sports, etc., and is accompanied by the scene judgment confidence (e.g., "shopping mall: 0.96"). When the confidence is <0.8, historical data smoothing is enabled (taking the majority result of the first 3 frames) to ensure the accuracy of scene identification.

[0043] For example, the processing device can first input a set of fused observation sequences (corresponding to multi-dimensional features, including positional changes, obstacle distribution density, gait coherence, ambient light intensity, heart rate change trends, etc.) extracted every 500ms into a preset scene recognition model: Then, the CNN layer in the scene recognition model first extracts the spatial features of the fused observation sequence, such as the distribution pattern of Bluetooth beacon signals in indoor scenes, the density features of GPS trajectories in outdoor scenes, and the distribution characteristics of vibration frequencies in transportation scenes; then, the LSTM layer in the fused observation sequence captures the temporal features, such as the continuity of user gait changes within 1 second and the fluctuation of position movement rate, forming a "spatial-temporal" fused feature vector. Finally, the classifier in the scene recognition model classifies the "spatial-temporal" fused feature vector, outputs the current scene identifier and confidence score, such as "outdoor-park: 0.98", and determines the final scene recognition result based on the preset confidence threshold (such as 0.8) and confidence score ranking. If a classification signal with a confidence score < 0.8 (such as "transportation-bus: 0.75") appears, historical data is used for data smoothing, and the majority of scene identifiers in the top N (N greater than or equal to 3) are taken as the final determined current scene identifier.

[0044] It should be noted that in this embodiment, the sensor confidence and scene recognition in the fusion decision layer are not "separate" but rather operate in parallel and collaboratively. That is, during the confidence determination process, the weight adjustment of the weighted Kalman filter can directly refer to the real-time / most recent output of the scene recognition process (scene recognition updates every 500ms, sensor data is collected every 100ms, and the latest scene result can be reused during fusion). For example, if the scene recognition determines that it is "indoors," the GPS weight will be immediately reduced and the Bluetooth beacon weight will be increased. At the same time, when the fusion decision layer processes sensor data every 100ms, it will synchronously call the latest scene conclusion output by the scene recognition process, use scene information to guide weight allocation, and then output the fused state data to ensure the consistency between scene adaptation and data fusion.

[0045] Step S30: Generate multimodal feedback information based on the current scene identifier.

[0046] It is easy to understand that the aforementioned multimodal feedback information can be a prompt signal output based on the current scene identifier, which can precisely control various feedback methods such as voice, vibration, and Braille dots. In this embodiment, the processing device can dynamically adjust the feedback prompt form according to the information priority corresponding to the current scene identifier and the scene requirements, and generate corresponding multimodal feedback information.

[0047] In this embodiment, to address the problems of functional fragmentation (such as navigation and obstacle warning operating independently) and poor scene adaptability (the same function has the same effect in different scenarios without targeted optimization) in existing assistive devices for visually impaired users, resulting in low assistive efficiency and poor practicality for visually impaired individuals, this application integrates multi-source sensor data for information processing to eliminate the problem of single sensor susceptibility to interference (such as inaccurate GPS indoor positioning and large detection errors of distance sensors under strong light), thereby improving the accuracy of environmental and user status judgment; and accurately acquires the current scene by using a scene recognition model, allowing multimodal feedback information to adapt to scene requirements, reducing the operational burden on visually impaired users, and improving the accuracy and ease of use of the assistive device.

[0048] In one feasible implementation, refer to Figure 3 , Figure 3 This is a schematic diagram of the third process of the first embodiment of the adaptive information processing method of this application. In this embodiment, after step S10, steps S40 to S50 may also be included: Step S40: When a preset abnormal signal is detected in the fused observation sequence, obtain the abnormal event identifier corresponding to the preset abnormal signal; Step S50: Generate early warning response information based on the abnormal event identifier.

[0049] It is easy to understand that the aforementioned preset abnormal signals can be signals in the fused observation sequence that meet the preset abnormal conditions, including obstacle abnormal signals (corresponding to obstacles with a distance of <1 meter and being dynamic obstacles), physiological abnormal signals (corresponding to user heart rate >150 beats / minute for 1 minute or blood oxygen <90% for 30 seconds), and fall abnormal signals (corresponding to a sudden change in user movement acceleration detected by the motion sensor >5g and a change in posture angle >90° (characterizing the user's body changing from vertical to horizontal), where g is the acceleration due to gravity). The preset abnormal conditions can be set based on safety standards and user health data.

[0050] Therefore, in this embodiment, the abnormal events corresponding to the preset abnormal signals can be classified and identified, that is, the above-mentioned abnormal event identification. For example, the obstacle abnormal signal (e.g., distance 0.8 meters, dynamic obstacle) can be identified as "emergency obstacle - dynamic obstacle"; the physiological abnormal signal (e.g.) can be identified as "physiological emergency - high heart rate / low blood oxygen"; and the fall abnormal signal can be identified as "safety emergency - fall". The abnormal event identification can include the abnormal type and the degree of urgency.

[0051] The aforementioned early warning response information can be information generated in advance for abnormal event identifiers, including early warning prompts and emergency handling operations. It is easy to understand that, in addition to conventional multimodal feedback, the early warning response information can also include emergency response information such as notification of emergency contacts, audible and visual alarms, and automatic dialing (only for high-urgency abnormal events), to ensure that users receive timely assistance.

[0052] For example, for an obstacle abnormality event (identified as "emergency obstacle - dynamic obstacle"), the processing device can generate a corresponding warning response information of "urgent voice prompt ('There is a dynamic obstacle 0.8 meters ahead, take emergency avoidance', played in a loop until the user confirms) + vibration (frequency 8Hz, intensity level 5, lasting for 5 seconds, repeated at 1-second intervals) + Braille dot matrix (dynamic obstacle 0.8 meters ahead, continuously displayed)". At the same time, based on the real-time functional strategy set, the CPU utilization rate of the obstacle warning module is temporarily increased to 50% to ensure that the feedback is continuously effective.

[0053] Regarding physiological emergencies (identified as "physiological emergency - high heart rate"), the processing device can generate corresponding warning response information, including "voice prompt ('Current heart rate 160 beats / minute, lasting for 1 minute, exceeds the safe range, it is recommended to stop activity immediately and sit down to rest', played in a loop) + vibration (frequency 5Hz, intensity level 3, lasting for 3 seconds, repeated every 2 seconds) + Braille dot matrix ('Heart rate 160 emergency, please rest immediately', continuously displayed)". At the same time, it automatically retrieves emergency contact information from the personalized configuration data (such as the user's preset "family member A, phone number 138XXXX1234") and sends an SMS message containing the user's current location to the emergency contact ("User's current heart rate is abnormal (160 beats / minute), location: near No. XX, XX Street, XX District, XX City, please pay attention"). If the heart rate does not drop to the safe range within 10 minutes, a reminder is sent again.

[0054] For safety emergencies (marked "Safety Emergency - Fall"), the processing device can generate a corresponding warning response message: "Local audible and visual alarm (high-decibel prompt tone 100 decibels, flashing light frequency 5Hz, continuously on) + voice confirmation ('Do you need help? If there is no response within 10 seconds, we will automatically contact the emergency contact') + vibration (frequency 10Hz, intensity level 5, lasting 10 seconds) + Braille dot matrix ('Fall, need help', continuously displayed)". If there is no user response within 10 seconds (such as touching the confirmation button), the processing device can automatically call the emergency contact (prioritizing the preset first contact). After the call is connected, a preset voice message will be played ('The user may have fallen. Please contact us or go to the rescue as soon as possible. Current location: near No. XX, XX Street, XX District, XX City') and a location SMS will be sent at the same time.

[0055] In addition, the processing device can also monitor motion information in the user status data. If the user's stillness time reaches a preset threshold, it will automatically enter sleep mode, that is, control some sensors and functional modules to sleep, retain the voice wake-up function, and reduce power consumption by 60%.

[0056] In this embodiment, preset abnormal signal detection ensures timely detection of emergency anomalies, preventing overlooked dangers. Pre-set abnormal event identifiers clearly define the anomaly type and urgency level, providing a basis for emergency response. Finally, the warning response information generated by the processing device can be combined with routine feedback and emergency handling (emergency contact notification, audible and visual alarms, automatic dialing), not only informing users of anomalies promptly but also enabling them to receive external assistance when they are unable to call for help independently. This significantly improves device safety and provides dual protection for the travel and health of visually impaired users.

[0057] This embodiment provides a scene-adaptive information processing method, which includes: performing pre-standardized processing on multi-source sensor data to obtain standard sensor data; performing spatiotemporal alignment preprocessing on the standard sensor data to obtain standardized sub-sequences; weighted fusion of the standardized sub-sequences based on real-time sensor confidence to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data; inputting the fused observation sequence into a scene recognition model to obtain a current scene identifier; and generating multimodal feedback information based on the current scene identifier. When a pre-set abnormal signal is detected in the fused observation sequence, an abnormal event identifier corresponding to the pre-set abnormal signal is obtained; and a warning response information is generated based on the abnormal event identifier. This embodiment solves the problem of functional fragmentation through multi-sensor fusion and scene recognition, improves the accuracy of environmental perception and the timeliness of feedback, adapts to different scene requirements, and reduces the operational burden on visually impaired users. At the same time, real-time anomaly detection and automatic response to abnormal events improve safety and reduce intervention delays.

[0058] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment described above can be referred to the above description, and will not be repeated hereafter.

[0059] Based on the first embodiment, please refer to Figure 4 , Figure 4 This is a schematic diagram of the first process of the second embodiment of the adaptive information processing method of this application. In this embodiment, step S30 further includes steps S31 to S32: Step S31: Determine the set of real-time functional strategies corresponding to the preset functional modules based on the current scene identifier; It is easy to understand that the aforementioned set of real-time functional strategies can be a set of operating strategies formulated for preset functional modules based on the current scene identifier. This may include module operating priorities (e.g., the outdoor scene navigation module has higher priority than the physiological monitoring module), resource allocation schemes (CPU utilization, sensor sampling frequency, e.g., increasing the CPU utilization of core functional modules to 30% and increasing the sampling frequency to 100ms / time), and function triggering conditions (e.g., the obstacle warning module triggers a prompt when the obstacle distance is <3 meters). The processing device can ensure that the preset functional modules are adapted to the user's current scene based on the set of real-time functional strategies, improving the accuracy of device assistance. At this time, the multimodal feedback information generated by the processing device can be dynamically adjusted according to the functional strategies (e.g., higher priority functions correspond to more urgent feedback methods).

[0060] The aforementioned preset functional modules can be a collection of software modules in the processing device that implement specific auxiliary functions. These modules may include a navigation module that provides location navigation based on GPS or Bluetooth beacons, an obstacle warning module that detects obstacles and provides alerts based on distance sensor data, a Bluetooth positioning module that locates based on Bluetooth beacons in indoor scenarios, a scene description module that describes the current scene environment via voice, a physiological monitoring module that monitors physiological indicators such as heart rate and blood oxygen, and a gait correction module that prompts the user to adjust their gait. Each module can operate independently or work in concert.

[0061] As one possible implementation, in this embodiment, step S31 includes steps B1~B2: Step B1: Convert the current scene identifier into the current weight vector according to the preset strategy mapping table; Step B2: Based on the current weight vector, prioritize the operation of the navigation module, obstacle warning module, Bluetooth positioning module, scene description module, physiological monitoring module, and gait correction module to form a real-time functional strategy set.

[0062] It is understood that the aforementioned current weight vector can be a vector composed of the weight values ​​of each preset functional module in the current scenario, and the order of the vector elements (which can be determined according to the actual situation, and this embodiment does not impose any restrictions on this) corresponds to the preset functional modules currently mounted on the processing device (e.g., navigation → obstacle warning → Bluetooth positioning → scene description → physiological monitoring → gait correction), which can be directly used for function priority ranking. The aforementioned preset strategy mapping table can be a table that stores the mapping relationship between scene identifiers and the current weight vector, and each scene identifier in the table corresponds to a set of functional weight vectors. Each element in the vector represents the weight value of a preset functional module (which can take values ​​from 0 to 10, with larger values ​​indicating that the module is more important in the scenario). The specific weight values ​​are determined based on scenario requirements and user needs research (e.g., the weight of the navigation module in an outdoor scene is 10, and the weight of the Bluetooth positioning module in an indoor scene is 8), and this embodiment does not impose any restrictions on this.

[0063] For example, in an outdoor scene, the current weight vector can be [10 (navigation), 9 (obstacle warning), 3 (Bluetooth positioning), 7 (scene description), 5 (physiological monitoring), 4 (gait correction)]; in an indoor scene, the current weight vector can be [5, 7, 8, 10, 6, 3]; in a sports scene, the current weight vector can be [|3, 4, 1, 2, 10, 9]; in a transportation scene, the current weight vector can be [6, 6, 7, 9, 5, 2].

[0064] Therefore, the processing device can prioritize the preset functional modules according to the weight values ​​of each module in the current weight vector (the larger the weight value, the higher the priority). The ranking result serves as the core content of the real-time functional strategy set, guiding the allocation of module resources and the triggering of functions. For example, the ranking result in a motion scenario can be: physiological monitoring module (10) > gait correction module (9) > obstacle warning module (4) > navigation module (3) > scene description module (2) > Bluetooth positioning module (1). At the same time, the ranking result can be marked as priority 1 to 6 (physiological monitoring is 1, Bluetooth positioning is 6). Among them, core functional modules ranked higher receive preferential system resources (such as increased CPU utilization and higher sensor sampling frequency), and are given priority in responding to command conflicts (e.g., in outdoor scenarios, navigation commands have higher priority than health data broadcasts). Non-core functional modules ranked lower have their operating power reduced by the controller (e.g., longer sampling intervals and low priority background operation), ensuring that they can still be activated normally when triggered by the user (e.g., health monitoring sampling is changed from 100ms / time to 500ms / time in outdoor scenarios). For example, in transportation scenarios, core functions (such as voice broadcasts of stations and vibration reminders to get off the vehicle corresponding to the scene description module) are given priority, while non-core functions (such as gait correction) operate at a reduced frequency, ensuring that core needs are met while maintaining functional integrity. In outdoor scenarios, GPS navigation and obstacle warnings can be enhanced (response latency ≤ 0.5 seconds). In indoor scenarios, Bluetooth beacon positioning and voice scene descriptions can be activated (e.g., "There is an elevator 5 meters ahead, and there is a handrail on the left"). In sports scenarios, physiological data monitoring and gait correction prompts can be prioritized (e.g., "Stride is too large, it is recommended to slow down").

[0065] Therefore, if the scene is identified as an outdoor scene, the processing device can enhance the positioning and navigation function and obstacle warning function; if the scene is identified as an indoor scene, the processing device can prioritize the short-range positioning function and scene voice description function; if the scene is identified as a sports scene, the processing device can prioritize the physiological data monitoring function and gait prompt function; if the scene is identified as a transportation scene, the processing device can activate the stability monitoring and arrival reminder function.

[0066] Furthermore, the corresponding real-time function strategy set for sports scenarios can be as follows: resources are allocated to each module based on priority: Priority 1 (physiological monitoring) CPU utilization 35%, sampling frequency 100ms / time, trigger condition is heart rate > 120 beats / minute or blood oxygen < 95%; Priority 2 (gait correction) CPU utilization 30%, sampling frequency 100ms / time, trigger condition is stride fluctuation > 0.15 meters or gait cycle fluctuation > 0.2 seconds; Priority 3 (obstacle warning) CPU utilization 15%, sampling frequency 200ms / time, trigger condition is obstacle distance < 2 meters; Priority 4 (navigation) CPU utilization 10%, sampling frequency 500ms / time, trigger condition is deviation from the running route > 5 meters; Priority 5 (scene description) CPU utilization 5%, sampling frequency 1000ms / time, trigger condition is entering a new running area (such as entering the track from the playground); Priority 6 (Bluetooth positioning) CPU utilization 5%, sampling frequency 2000ms / time, only enabled when the navigation signal is weak. By integrating the real-time priorities, resource allocations, and function triggering conditions corresponding to the above modules, a set of real-time function strategies for motion scenarios can be formed. The generation process of real-time function strategy sets for other scenarios is similar and will not be repeated in this embodiment.

[0067] In this embodiment, the existing preset priority ranking of functional modules lacks a clear basis and relies heavily on manual experience, resulting in a mismatch between the ranking results and scenario requirements (e.g., in a sports scenario, the physiological monitoring module has a lower priority than the navigation module). Furthermore, resource allocation and triggering conditions are not associated with priority, leaving core modules without effective support in specific scenarios. This embodiment establishes a clear correspondence between scenario identifiers and current weight vectors through a preset strategy mapping table, ensuring that the weight vectors meet scenario requirements. The priority ranking based on the weight vectors is objective and accurate, granting high priority to core modules (such as physiological monitoring and gait correction in sports scenarios). Resource allocation and triggering conditions are dynamically adjusted according to priority, ensuring efficient operation of core modules and improving the device's targeted and effective assistance in specific scenarios.

[0068] Step S32: Generate multimodal feedback information through the real-time functional strategy set.

[0069] It is easy to understand that the processing device can generate information including multiple feedback methods such as voice, vibration, and Braille dot matrix when necessary, based on the set of real-time function strategies and combined with environmental and user status data in the fusion observation sequence. The feedback method and content can be dynamically adjusted according to the real-time function strategy (such as higher priority functions corresponding to more urgent feedback methods).

[0070] For example, if the current scene is identified as "outdoor-street", the navigation module has the highest priority in the real-time function strategy set. If the processing device determines that the user's current position deviates from the preset route by 1.5 meters based on the fused observation sequence (triggering the navigation prompt condition), the navigation module can generate "voice prompt ('You have deviated from the route, please turn 30 degrees to the right and walk straight for 20 meters along the current road to return to the route') + vibration feedback (1 long vibration, indicating that the direction needs to be adjusted)"; if a dynamic obstacle (pedestrian) is detected 2.5 meters ahead, triggering the warning condition, the obstacle warning module can generate The system includes a rapid voice prompt ("Pedestrian ahead 2.5 meters, please be careful") + vibration feedback (3 rapid vibrations) + Braille dot matrix display ("↑Person 2.5 meters away"); if the system detects that the user has entered an alley (triggering the environmental description condition), the scene description module can generate a voice description ("Currently entering XX alley, the road is narrow, there is a trash can on the left"); the physiological monitoring module can normally display a heart rate of 85 beats / minute (normal range 60-100 beats / minute, no prompt triggered); the gait correction module can display a stride of 0.7 meters (stable, no prompt triggered). Then, the feedback information from each module is integrated to form multimodal feedback information (output in order of priority, with emergency feedback information being output in advance, such as obstacle warning feedback taking precedence over scene description feedback).

[0071] As one possible implementation, in this embodiment, step S32 includes steps C1~C2: Step C1: Obtain the early warning level corresponding to the fused observation sequence; Step C2: Generate multimodal feedback information based on the warning level and the set of real-time functional strategies.

[0072] It should be understood that the aforementioned warning levels can be classified according to the urgency of abnormal events (such as obstacle distance, obstacle type, abnormal physiological indicators, and falls) in the fused observation sequence. In this embodiment, they can be divided into emergency (level 1), warning (level 2), and alert (level 3), with the urgency decreasing from high to low. The level classification can be determined based on the preset thresholds of different signals in the warning level threshold table. For example, obstacle distance < 1 meter is level 1, 1-3 meters is level 2, and 3-5 meters is level 3; heart rate > 150 beats / minute is level 1, 120-150 beats / minute is level 2, and 100-120 beats / minute is level 3.

[0073] At this point, the processing device can use the warning level as the basis for judging the urgency of the feedback information. Combined with the module priority and triggering conditions in the real-time function strategy set, it can adjust the feedback method (e.g., level 1 corresponds to "urgent voice prompt + high-frequency vibration + continuous Braille dot matrix display", level 2 can correspond to "urgent voice prompt + slight vibration", and level 3 can correspond to "voice prompt") and output order (level 1 feedback takes precedence over other levels), and generate multimodal feedback information with corresponding real-time feedback method and output order to ensure that emergency abnormal events are perceived by users first.

[0074] Furthermore, taking an "outdoor-street" scenario (obstacle warning level 1, physiological warning level 3) as an example: the processing device can first process the level 1 obstacle warning, generating obstacle warning feedback information based on the obstacle warning module trigger conditions (distance < 3 meters) and feedback rules (level 1 corresponds to "rapid voice prompt (frequency 2Hz, maximum volume) + vibration (frequency 5Hz, maximum intensity, lasting 3 seconds) + Braille dot matrix ('car ahead 0.8 meters', continuously displayed until user feedback)" in the real-time functional strategy set); then it processes the level 3 physiological warning, generating physiological warning feedback information based on the physiological monitoring module trigger conditions (heart rate 100-120 beats / minute) and feedback rules (level 3 corresponds to "smooth voice prompt ('current heart rate 110 beats / minute, slightly higher than normal, please rest') + Braille dot matrix display ('heart rate 110', lasting 2 seconds)"). The feedback information output order is obstacle warning feedback first, then physiological warning feedback, ensuring that the user is aware of the emergency anomaly first.

[0075] Taking the "indoor-home" scenario (physiological warning level 2, obstacle warning level 2) as an example: the two warning levels are the same, and are sorted according to the priority of the modules in the real-time functional strategy set (assuming that the physiological monitoring module in the indoor scenario has a priority of 3 and the obstacle warning module has a priority of 2). The obstacle warning with higher priority is processed first. Level 2 corresponds to "voice prompt ('There is furniture 2 meters ahead, please detour', frequency 1.5Hz) + vibration (frequency 5Hz, intensity moderate, lasting 5 seconds)"; then the physiological warning is processed. Level 2 corresponds to "voice prompt ('Current heart rate is 140 beats / minute, higher than the normal range, it is recommended to sit down and rest', frequency 1.5Hz) + vibration (frequency 3Hz, intensity moderate, lasting 2 seconds)".

[0076] In this embodiment, existing devices generate feedback information without considering the urgency of abnormal events, only outputting based on module priority. This results in the feedback of urgent abnormal events (such as a vehicle 1 meter ahead) being delayed by non-urgent information (such as a slightly elevated heart rate), preventing users from avoiding danger in time. Furthermore, the output order of feedback information of the same urgency level is disordered, affecting the efficiency of user information reception. This embodiment clarifies the urgency of abnormal events by classifying warning levels, prioritizing the output of feedback for urgent events to ensure users can perceive danger promptly. It also combines the module priority of the real-time function strategy set to solve the problem of the output order of warning information of the same level, ensuring that important modules receive feedback first, further improving the effectiveness and security of feedback information, and providing more reliable protection for users' travel and health.

[0077] As one possible implementation method, refer to Figure 5 , Figure 5 This is a schematic diagram of the third process of the second embodiment of the adaptive information processing method of this application. In this embodiment, step C2 includes steps C21 to C23: Step C21: Obtain personalized configuration data and historical user behavior data; Step C22: Optimize and identify the monitored user interaction operations based on historical user behavior data to obtain the user's operation intent; Step C23: Generate multimodal feedback information based on personalized configuration data, user operation intent, warning level, and real-time functional strategy set.

[0078] Understandably, the aforementioned personalized configuration data can be user-specific configuration data stored in the processing device, which may include voice configuration (speech speed 50-150 words / minute, volume level 1-10, voice type (male / female / child's voice)), vibration configuration (vibration intensity level 1-5, vibration frequency 1-10Hz), Braille dot matrix configuration (display brightness level 1-3, display duration 1-5 seconds), and personalized adjustment of warning thresholds (e.g., if the user's normal heart rate is low, lower the heart rate warning threshold to 90-130 beats / minute), etc., which can be set by the user upon first use or manually modified later. The aforementioned historical user behavior data can be records of the user's past device usage, including the user's response to feedback information (e.g., whether to touch the processing device to confirm after hearing a voice prompt, and the adjustment of vibration feedback), the frequency of use of commonly used function modules (e.g., the user uses the navigation module 5 times a day and the physiological monitoring module 3 times a day), and command interaction records (e.g., the user's commonly used voice commands "query time" and "navigate to supermarket"). The data is stored by timestamp and updated periodically (e.g., monthly).

[0079] It should be understood that the above-mentioned user interaction operations can be interactive actions performed by the user with the processing device through voice, touch (touch of the Braille dot matrix area), such as the user inputting the voice command "What's ahead?" or touching the "confirm" button in the Braille dot matrix area, so that the processing device can recognize the user's needs through the user interaction operations.

[0080] At this point, the processing device can analyze the user's true needs based on historical user behavior data and current user interaction. For example, if a user says "go there" in an outdoor scene, and the user frequently navigates to "XX supermarket" in historical behavior, the user's intention can be inferred to be "navigate to XX supermarket". If the user touches the "previous page" button in the Braille dot matrix area (user interaction), the device can optimize the recognition of the user's intention to "replay the most recent scene description" by combining historical data showing that there is a 70% probability that the user wants to listen to the previous scene description again after touching "previous page".

[0081] At this point, the processing device can adaptively adjust each functional module based on personalized configuration data, such as adjusting the trigger threshold, generating an optimized set of real-time functional strategies, and combining the identified user operation intent with the optimized set of real-time functional strategies to generate high-precision multimodal feedback information.

[0082] For example, taking the "outdoor-street" scenario (obstacle warning level 1, user intent "navigate to XX supermarket") as an example: in the real-time function strategy set, the navigation module has priority 1 and the obstacle warning module has priority 2; in the personalized configuration data, the voice speed is 100 words / minute and the vibration intensity is level 3; warning level 1 corresponds to emergency feedback. First, obstacle warning feedback can be generated: according to personalized configuration, the voice prompts "Vehicle ahead 0.8 meters, emergency avoidance" at a speed of 100 words / minute and a volume of 7, the vibration is at a strength of 3 and a frequency of 5Hz for 3 seconds, and the Braille dot matrix displays "Vehicle ahead 0.8 meters" at a brightness of 2 for 3 seconds; then, navigation feedback can be generated: combined with the recognized user intent "Navigate to supermarket C", the voice prompts "Navigation route has been planned for you at a speed of 100 words / minute, turn right from the current location, go straight along street A for 500 meters, turn left onto road B, and go straight for 300 meters to reach supermarket C", the vibration is at a strength of 3 and a frequency of 2Hz once (indicating the start of navigation information), and the Braille dot matrix displays "Navigation: turn right → street A 500 meters → turn left onto road B 300 meters → supermarket C" for 3 seconds. All feedback conforms to personalized configuration and scene strategy, and emergency warnings are output with priority.

[0083] Therefore, existing auxiliary devices use a uniform configuration for feedback information, failing to consider personalized user needs (such as user speech rate preferences and vibration sensitivity). Furthermore, they suffer from low accuracy in recognizing ambiguous user interactions (such as ambiguous voice commands), resulting in a poor match between feedback information and user needs, and a poor user experience. This embodiment addresses these issues by using personalized configuration data to adapt feedback information to user habits, improving user comfort (e.g., speech rate matching user auditory habits); optimizing the recognition of user operation intentions based on historical behavioral data to improve the accuracy of ambiguous command recognition, ensuring that feedback information meets the user's actual needs; and finally, combining warning levels and scenario strategies to ensure that the generated multimodal feedback information satisfies personalized needs while prioritizing emergency information, significantly improving device usability and user experience.

[0084] In summary, existing assistive devices for the blind suffer from problems such as fixed operating strategies for each functional module, which fail to dynamically adjust according to the scenario. This leads to excessive resource consumption for core functions (e.g., outdoor navigation) in non-core scenarios (e.g., indoors), while resources are insufficient in core scenarios. Furthermore, the timing of function triggering and feedback methods do not meet scenario requirements, resulting in low device efficiency. This embodiment addresses these issues by using a real-time functional strategy set determined by the current scenario identifier. This allows the operating priority and resource allocation of each functional module to adapt to scenario requirements, ensuring that core functions receive sufficient resources in their respective scenarios and guaranteeing efficient operation. By combining personalized user configurations, high-precision user recognition intent, and multimodal feedback information generated from the real-time functional strategy set, the embodiment ensures that the feedback content and methods not only meet scenario requirements but also provide high-efficiency prompts that align with user habits. This improves the efficiency of visually impaired users in receiving feedback information and further enhances the device's practicality.

[0085] In this embodiment, the current scene identifier is converted into a current weight vector according to a preset strategy mapping table. Based on the current weight vector, the functions of the navigation module, obstacle warning module, Bluetooth positioning module, scene description module, physiological monitoring module, and gait correction module are prioritized to form a real-time function strategy set. The warning level corresponding to the fused observation sequence is obtained; personalized configuration data and historical user behavior data are obtained; based on the historical user behavior data, the monitored user interaction operations are optimized and identified to obtain the user's operation intent; multimodal feedback information is generated based on the personalized configuration data, user operation intent, warning level, and real-time function strategy set. This embodiment can adapt the running priority and resource allocation of each functional module to the scene requirements based on the real-time function strategy set determined by the current scene identifier, ensuring that core functions receive sufficient resources in the corresponding scene and guaranteeing efficient operation. Furthermore, by combining the user's personalized configuration, the generated high-precision user identification intent, and the real-time function strategy set to generate multimodal feedback information, it ensures that the feedback content and method not only meet the scene requirements but also have high prompting efficiency and conform to user habits, improving the efficiency of visually impaired users in receiving feedback information and further enhancing the practicality of the device.

[0086] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the adaptive information processing method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0087] This application also provides a scene adaptive information processing device, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the module structure of the scene adaptive information processing device according to an embodiment of this application. In this embodiment, the scene adaptive information processing device includes: The data processing module 601 is used to preprocess multi-source sensor data to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data. Scene recognition module 602 is used to input the fused observation sequence into the scene recognition model to obtain the current scene identifier; The adaptive feedback module 603 is used to generate multimodal feedback information based on the current scene identifier.

[0088] As one possible implementation, in this embodiment, the data processing module 601 is further configured to perform preset standardization processing on multi-source sensor data to obtain standard sensor data; perform spatiotemporal alignment preprocessing on the standard sensor data to obtain standardized sub-sequences; and perform weighted fusion of the standardized sub-sequences based on the real-time confidence of the sensors to obtain fused observation sequences.

[0089] As one possible implementation, in this embodiment, the adaptive feedback module 603 is further used to determine the set of real-time functional strategies corresponding to the preset functional modules based on the current scene identifier; and to generate multimodal feedback information through the set of real-time functional strategies.

[0090] As one possible implementation, in this embodiment, the preset functional modules include: a navigation module, an obstacle warning module, a Bluetooth positioning module, a scene description module, a physiological monitoring module, and a gait correction module; the adaptive feedback module 603 is further used to convert the current scene identifier into a current weight vector according to a preset strategy mapping table; and to sort the operation priority of the functions of the navigation module, obstacle warning module, Bluetooth positioning module, scene description module, physiological monitoring module, and gait correction module based on the current weight vector, forming a real-time functional strategy set.

[0091] As one possible implementation, in this embodiment, the adaptive feedback module 603 is also used to obtain the warning level corresponding to the fused observation sequence; and to generate multimodal feedback information based on the warning level and the real-time functional strategy set.

[0092] As one possible implementation, in this embodiment, the adaptive feedback module 603 is also used to acquire personalized configuration data and historical user behavior data; optimize and identify the monitored user interaction operations based on the historical user behavior data to obtain the user's operation intent; and generate multimodal feedback information based on personalized configuration data, user operation intent, warning level, and real-time functional strategy set.

[0093] As one possible implementation, in this embodiment, the adaptive feedback module 603 is further configured to obtain the abnormal event identifier corresponding to the preset abnormal signal when a preset abnormal signal is detected in the fused observation sequence; and generate early warning response information based on the abnormal event identifier.

[0094] The scene adaptive information processing apparatus provided in this application, employing the scene adaptive information processing method in the above embodiments, can solve the technical problem of scene adaptive information processing. Compared with the prior art, the beneficial effects of the scene adaptive information processing apparatus provided in this application are the same as those of the scene adaptive information processing method provided in the above embodiments, and other technical features in the scene adaptive information processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0095] This application provides a scene adaptive information processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the scene adaptive information processing method in the above embodiment 1.

[0096] The following is for reference. Figure 7 This document illustrates a structural schematic diagram of a scene-adaptive information processing device suitable for implementing embodiments of this application. The scene-adaptive information processing device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), smartwatches, and fixed terminals such as digital TVs and desktop computers. Figure 7 The scene-adaptive information processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0097] like Figure 7 As shown, the scene-adaptive information processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the scene-adaptive information processing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the scene adaptive information processing device to communicate wirelessly or wiredly with other devices to exchange data. Although a scene adaptive information processing device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0098] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment disclosed in this application includes a scene adaptive information processing program product, which includes a scene adaptive information processing program carried on a computer-readable medium, the scene adaptive information processing program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the scene adaptive information processing program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the scene adaptive information processing program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0099] The scene-adaptive information processing device provided in this application, employing the scene-adaptive information processing method in the above embodiments, can solve the technical problem of how to improve the accuracy and ease of use of device assistance and effectively reduce the operational burden on visually impaired people. Compared with the prior art, the beneficial effects of the scene-adaptive information processing device provided in this application are the same as those of the scene-adaptive information processing method provided in the above embodiments, and other technical features in this scene-adaptive information processing device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0100] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0101] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0102] This application provides a storage medium having computer-readable program instructions (i.e., a scene adaptive information processing program) stored thereon, which are used to execute the scene adaptive information processing method in the above embodiments.

[0103] The storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of the storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0104] The aforementioned storage medium may be included in the scene adaptive information processing device; or it may exist independently and not be assembled into the scene adaptive information processing device.

[0105] The aforementioned storage medium carries one or more programs. When the aforementioned one or more programs are executed by the scene adaptive information processing device, the scene adaptive information processing device becomes: scene adaptive information processing.

[0106] The scenario-adaptive information processing program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of system, method, and scenario-adaptive information processing program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0108] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0109] The readable storage medium provided in this application is a storage medium that stores computer-readable program instructions (i.e., a scene adaptive information processing program) for executing the above-described scene adaptive information processing method. This solves the technical problem of how to improve the accuracy and ease of use of assistive devices and effectively reduce the operational burden on visually impaired individuals. Compared with the prior art, the beneficial effects of the storage medium provided in this application are the same as those of the scene adaptive information processing method provided in the above embodiments, and will not be repeated here.

[0110] The above are only some embodiments of this application and do not limit the scope of the solution of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.

Claims

1. A scene adaptive information processing method, characterized in that, The method includes: The multi-source sensor data is preprocessed to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data. The fused observation sequence is input into the scene recognition model to obtain the current scene identifier; Multimodal feedback information is generated based on the current scene identifier.

2. The scene adaptive information processing method as described in claim 1, characterized in that, The step of preprocessing multi-source sensor data to obtain a fused observation sequence includes: Multi-source sensor data is subjected to pre-defined standardization processing to obtain standard sensor data; Perform spatiotemporal alignment preprocessing on the standard sensor data to obtain standardized subsequences; The standardized subsequences are weighted and fused based on the real-time confidence scores of the sensors to obtain a fused observation sequence.

3. The scene adaptive information processing method as described in claim 1, characterized in that, The step of generating multimodal feedback information based on the current scene identifier includes: Based on the current scene identifier, determine the set of real-time function strategies corresponding to the preset function modules; Multimodal feedback information is generated through the set of real-time functional strategies.

4. The scene adaptive information processing method as described in claim 3, characterized in that, The preset functional modules include: a navigation module, an obstacle warning module, a Bluetooth positioning module, a scene description module, a physiological monitoring module, and a gait correction module; The step of determining the set of real-time function strategies corresponding to the preset function modules based on the current scene identifier includes: The current scene identifier is converted into the current weight vector according to the preset strategy mapping table; Based on the current weight vector, the functions of the navigation module, the obstacle warning module, the Bluetooth positioning module, the scene description module, the physiological monitoring module, and the gait correction module are prioritized to form a real-time function strategy set.

5. The scene adaptive information processing method as described in claim 3, characterized in that, The step of generating multimodal feedback information through the real-time function strategy set includes: Obtain the early warning level corresponding to the fused observation sequence; Multimodal feedback information is generated based on the warning level and the set of real-time functional strategies.

6. The scene adaptive information processing method as described in claim 5, characterized in that, The step of generating multimodal feedback information based on the warning level and the real-time functional strategy set includes: Obtain personalized configuration data and historical user behavior data; Based on the historical user behavior data, the monitored user interaction operations are optimized and identified to obtain the user's operation intent; Multimodal feedback information is generated based on the personalized configuration data, the user's operation intent, the warning level, and the set of real-time functional strategies.

7. The scene adaptive information processing method as described in claim 1, characterized in that, After preprocessing the multi-source sensor data to obtain the fused observation sequence, the process further includes: When a preset abnormal signal is detected in the fused observation sequence, the abnormal event identifier corresponding to the preset abnormal signal is obtained; Early warning response information is generated based on the abnormal event identifier.

8. A scene-adaptive information processing device, characterized in that, The scene adaptive information processing device includes: The data processing module is used to preprocess multi-source sensor data to obtain a fused observation sequence; the multi-source sensor data includes GPS positioning data, distance sensor data, motion sensor data, and physiological sensor data. The scene recognition module is used to input the fused observation sequence into the scene recognition model to obtain the current scene identifier; An adaptive feedback module is used to generate multimodal feedback information based on the current scene identifier.

9. A scene-adaptive information processing device, characterized in that, The device includes: a memory, a processor, and a scene adaptive information processing program stored in the memory and executable on the processor, the scene adaptive information processing program being configured to implement the steps of the scene adaptive information processing method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a scene adaptive information processing program, which, when executed by a processor, implements the steps of the scene adaptive information processing method as described in any one of claims 1 to 7.

Images