Mobile phone video communication warning system capable of displaying road condition ahead in real time

Abstract

The invention discloses a mobile phone video communication warning system capable of displaying the road condition in front in real time, and aims to solve the potential safety hazard during the walking of a'low-head family 'and the driving of an electric vehicle. The system comprises an image acquisition module, a GPS positioning and speed detection module, a data processing module, a road condition display module, a safety warning module and a video communication adaptation module. During working, the image acquisition module acquires a front road condition image in real time, and the GPS and speed module acquires a position and a moving speed; the data processing module analyzes the risk through a preset algorithm; if it is judged that the safety risk exists, the warning module gives out corresponding warning sound, meanwhile, the road condition display module displays road condition pictures on a mobile phone screen in an overlapped mode, and video communication is not interrupted. According to the system, parameters can be adaptively adjusted according to double scenes of walking and electric vehicle driving, the safety protection capability in the moving process of the low head family is effectively improved, and the practicability is high.

Description

Technical Field

[0001] This invention relates to the field of video communication technology, specifically to a mobile phone video communication warning system that can display real-time road conditions ahead. Background Technology

[0002] With the widespread use of smartphones, the number of "smartphone addicts" has surged. These addicts frequently cause accidents while walking or driving electric vehicles because they are focused on operating their phones and neglect road conditions. This has become a significant hidden danger to urban traffic safety.

[0003] Existing technologies for pedestrian and non-motorized vehicle safety protection are significantly inadequate. Firstly, traditional mobile phone navigation systems only provide route guidance and cannot capture real-time dynamic road conditions ahead. Furthermore, navigation prompts are disconnected from video communication scenarios, making it easy for users to ignore navigation voice prompts during video calls, thus negating the safety warning function. Secondly, some apps with a "pedestrian mode" can only determine whether the user is moving through the phone's sensors, failing to accurately identify the level of road risk. The warning signals are limited and have low visibility in noisy environments, making it difficult to effectively remind users to avoid danger. Thirdly, safety technologies for electric vehicle riding scenarios are even more lacking. Existing electric vehicle onboard devices mostly focus on range monitoring and anti-theft functions, without integration with mobile phone video communication functions. When users are making video calls while riding, the phone screen is occupied by the communication interface, making it impossible to observe road conditions ahead. Moreover, there is no dedicated risk calculation model based on riding speed and distance, making it impossible to dynamically adjust warning sensitivity according to riding speed—high false alarm rate at low speeds and delayed warnings at high speeds, both failing to meet safety requirements.

[0004] Furthermore, existing warning systems generally fail to consider the "situational specificity" of video communication: during video calls, users' attention is highly focused on the screen, significantly reducing their perception of the external environment. Conventional warning methods are easily ignored and may even interrupt the communication process, impacting user experience. In summary, existing technologies cannot simultaneously meet the communication and security needs of "smartphone addicts," necessitating an integrated system that can combine real-time traffic data collection, risk analysis, precise warnings, and communication adaptation. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a mobile phone video communication and warning system that can display real-time road conditions ahead, thus solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a mobile phone video communication warning system capable of displaying real-time road conditions ahead, comprising an image acquisition module, a GPS positioning and speed detection module, a data processing module, a road condition display module, a safety warning module, and a video communication adapter module. The modules interact with each other via the mobile phone's internal bus, as detailed below: The image acquisition module includes a wide-angle camera and a light compensation unit, which is used to acquire real-time road condition images 1-15m ahead during video communication. The light compensation unit dynamically adjusts the supplementary light according to the ambient light intensity to ensure that the image clarity meets the standard in low-light environments. The GPS positioning and speed detection module obtains position coordinates through the GPS positioning unit, and the inertial measurement unit collects acceleration and angular velocity to calculate the moving speed. Based on the speed and rate of change, it determines whether the scene is "walking" or "electric vehicle driving" and simultaneously sends the scene type to the data processing module. The data processing module extracts obstacle information using a lightweight image recognition algorithm, combined with movement speed. and scene coefficient Substitute into the risk level formula Assess the risks: <1 indicates no risk, only road condition images are transmitted; 1≤ <3 indicates low risk, triggering a low-risk warning and overlaying a yellow-bordered road condition image; A value of ≥3 indicates high risk, triggering a high-risk warning and displaying a road condition image with a red border and obstacle markers. The video communication adaptation module links with the native video communication APP through the API interface to overlay and merge road condition images with communication screens, with a fusion delay of ≤50ms and no interruption of communication. The traffic display module is based on the mobile phone screen and displays traffic information according to the principle of "communication screen as the main screen and traffic screen as the secondary screen". The transparency of the traffic screen can be adjusted. The safety warning module outputs a 1Hz, 60dB warning sound and a 2Hz vibration when the risk is low, and outputs a 2Hz, 75dB warning sound and a 4Hz vibration when the risk is high. The warning sound and communication volume can be adjusted independently.

[0007] Optionally, the image acquisition module further includes an obstacle dynamic tracking unit. After the image recognition algorithm detects an obstacle, this unit performs dynamic tracking using the inter-frame difference method: first, it extracts the pixel coordinates of the obstacle in two consecutive frames of images, and then calculates the pixel displacement. Combined with camera focal length Actual height of the obstacle and the pixel height of obstacles in the image Through formula ( (Real-time updates of obstacle distance, representing the change in distance between the obstacle and the phone) Simultaneously, the obstacle's movement direction is fitted using displacement trajectories from five consecutive frames. And calculate the movement speed. ( (frame interval time). Data processing module receives and Then, an obstacle dynamic correction coefficient is introduced. The risk level formula has been optimized and is now revised as follows: in The rule for determining the value is: when , =0.5; , =0.1; , =0.3; The dynamic tracking unit also sets tracking priorities for different obstacle types: pedestrians > moving vehicles > fixed obstacles. When multiple types of obstacles are detected at the same time, the risk level is calculated based on the higher priority obstacle to ensure more accurate response to dynamic dangerous targets.

[0008] Optionally, the GPS positioning and speed detection module further includes a scene verification unit, which ensures accurate scene type determination through a three-layer verification mechanism: The first layer is basic verification, which uses the speed and rate of change of the IMU to make a preliminary judgment on the scenario. The second layer is environmental verification, which uses GPS positioning data to call the map service API to obtain the road attributes of the current location, including whether it is a non-motorized vehicle lane, a pedestrian lane, or a mixed lane. If it is initially determined to be "electric vehicle driving" but the road attribute is "pedestrian lane", then a scenario questioning is triggered. The third layer is for device attitude verification, which uses the phone's gyroscope to collect tilt angles and calculates the angle fluctuation value within 10 seconds. Because mobile phones are often fixed to the handlebars when an electric vehicle is in motion, ≤5°; the handheld state of the mobile phone is unstable when walking. ≥15°; When the second layer of verification triggers a scenario challenge, if the third layer of verification... If the angle is ≥15°, the scene will be corrected to "walking"; if If the angle is ≤5°, the "electric vehicle driving" rule will be maintained, and the map service will be used to further confirm whether it is a "pedestrian crossing where electric vehicles are temporarily allowed". In this case, the scenario coefficient will be adjusted. Adjusted to 1.5.

[0009] Optionally, the safety warning module also includes a warning sound personalization unit, which provides multi-dimensional personalized configuration functions: Regarding the types of warning sounds, three basic libraries are preset: natural sound library, mechanical sound library, and voice prompt library. Users can select "low risk" and "high risk" respectively, and the system will push the appropriate type based on historical usage scenarios. The decibel range is set to 50dB-85dB, the vibration intensity is divided into 5 levels, and it supports a "smart adjustment" mode: it collects the ambient noise decibel value in real time through the mobile phone microphone. ,when When the noise level is ≥60dB, the warning sound will be automatically increased by 10dB and the vibration intensity will be increased by 1 level; when When the vibration level is ≤40dB, it will automatically decrease by 5dB and the vibration intensity will decrease by 1 level.

[0010] Optionally, the data processing module further includes an emergency warning unit, which presets extreme risk triggering conditions: in a walking scenario, the distance to the obstacle... ≤1m and moving speed ≥5km / h; in electric vehicle scenarios, ≤3m and ≥15km / h; or the obstacle type is "fast-moving vehicle" and When any of the above conditions are met, the emergency warning unit activates the "highest risk" mode: In addition to outputting high-risk warning sounds and vibrations, the safety warning module adds an "intermittent loud sound" and triggers the phone's flashlight to enhance the warning through multi-sensory stimulation; The video communication adaptation module automatically sends a two-layer prompt to the other party: the first layer is a floating text message "The other party is currently in a dangerous situation and cannot focus on communication at the moment," and the second layer is a 10-second automatic voice broadcast; at the same time, it forcibly switches the communication screen to "road condition priority display" and pauses the video capture on the user's end. When the distance to the obstacle returns to a safe value (walking) ≥2m, electric vehicle After ≥5m and 2 seconds, the system will automatically exit the "highest risk" mode and gradually restore the normal screen ratio and video capture. The communication connection will not be interrupted throughout the entire process.

[0011] Optionally, the system also includes a historical data storage module, which adopts a dual storage architecture of local and cloud storage. The local storage unit records the core data of each risk event, including event time, scenario type, risk level R value, obstacle type and coordinates, and user response behavior. The storage period is 30 days by default, and low-risk events are automatically deleted after the period expires. Cloud storage synchronizes local high-risk and emergency warning event data, using AES-256 encrypted transmission and supporting cross-device synchronization; Based on historical data, the system generates a weekly security report, which includes: the number of high-risk events and peak times each week, the distribution of high-frequency obstacle types, and the average user response time. The data processing module also optimizes scene coefficients using reinforcement learning algorithms. If a user fails to respond to a "high-risk event in a timely manner" three times consecutively in a certain scenario, the scenario will be automatically reclassified. The value will increase by 0.2; if there are 5 consecutive "low-risk event false alarms", then... The value decreased by 0.1.

[0012] Optionally, the video communication adaptation module also includes a multi-communication APP compatibility unit, which achieves cross-platform compatibility through a general interface and dedicated adaptation. The general interface layer adopts the AccessibilityService framework of Android and the NotificationCenter framework of iOS to monitor the currently active video communication APP process on the mobile phone in real time. When the APP starts a video call, the system's traffic collection and warning functions are automatically triggered. A dedicated adaptation layer is developed for mainstream apps using a dedicated SDK. It directly obtains communication video and audio streams through API interfaces and optimizes video blending efficiency based on the following formula: in, For the total fusion delay, Delay in rendering road condition images Delay for image overlay, These are the weighting coefficients; At the same time, by calling the APP's "picture-in-picture" permission, it is ensured that the traffic image can float on the top layer of the communication interface, without being restricted by the APP's own interface hierarchy. For the "voice call" scenario, the compatibility unit detects the call status through the telephone status monitoring interface. When it is determined that "a voice call is in progress" and the mobile phone screen is on, the traffic display and warning function is automatically activated. At this time, the traffic display occupies 1 / 5 of the screen to avoid obstructing the dialing interface. The compatibility unit allows users to manually add compatible apps. By selecting a non-preset communication app from the "App List", the system automatically analyzes its interface elements and generates adaptation rules.

[0013] Optionally, the traffic display module also includes a gesture and voice control unit, which provides multimodal interaction methods: For gesture control, three gestures and their recognition logic are preset: When two fingers are pinched together, if the distance between the two fingers decreases by ≥30% within 0.5 seconds, the traffic display will shrink, with each shrinkage being 10% of the current size, down to a minimum of 1 / 5 of the screen. Spread two fingers apart. If the distance between the two fingers increases by ≥30% within 0.5s, the road condition image will be magnified. Each magnification is 10% of the current size, up to a maximum of 1 / 3 of the screen. When a single finger touches the screen, if the movement trajectory is continuous and the direction is stable, and the movement distance is ≥1cm, a traffic display will be triggered and the screen will move with the finger. The movement range is limited to the four corners of the screen. The voice control integrates an offline voice recognition engine, with a preset command library including "zoom in on traffic conditions", "zoom out on traffic conditions", "hide traffic conditions", "show traffic conditions", and "switch location". The recognition process is as follows: (1) Collect user voice; (2) Extract speech feature vectors and match them with instruction library templates to calculate the matching degree. ; (3) When When the corresponding instruction is executed, a voice prompt indicating "executed" will be provided. (4) When When the user clicks "confirm," a confirmation pop-up window appears, waiting for the user to click "confirm" before execution. The unit also features a false touch protection mechanism: it automatically disables gesture control when it detects that the user is inputting text; and it raises the voice recognition threshold when the ambient noise level is ≥70dB. This reduces accidental triggering and ensures a balance between interaction efficiency and security.

[0014] This invention provides a mobile phone video communication and warning system that can display real-time road conditions ahead, and has the following beneficial effects: This mobile video communication and warning system, which can display real-time road conditions ahead, achieves integrated functions of "real-time road condition data collection, precise risk analysis, safety warnings, and communication adaptation" through multi-module collaboration and innovative algorithms. Compared with existing technologies, it has many outstanding advantages: The system uses a wide-angle camera and lightweight image recognition algorithm to capture dynamic road conditions in real time within a range of 1-15m. Combined with speed and location data from GPS and IMU, it constructs a multi-dimensional risk calculation model that integrates scene coefficients, speed, distance, and obstacle dynamics. The risk level determination latency is ≤100ms, which is faster than existing warning systems that rely solely on acceleration sensors. At the same time, it adaptively adjusts parameters for both "walking" and "electric vehicle driving" scenarios to match the safety requirements at different movement speeds, avoiding false alarms at low speeds and lag at high speeds. The system also achieves "overlay and fusion of road condition images and communication images" through a video communication adaptation module, which does not interrupt video communication and allows users to perceive road conditions while focusing on communication. In addition, the transparency, position, and size of the road condition images can be controlled by gestures or voice, the warning sound supports personalized settings, and the night mode automatically adjusts the warning mode to fully adapt to the diverse usage habits of users and avoid users abandoning the system due to complex operation or poor experience. This system is the first to integrate "dynamic obstacle tracking," "secondary scene verification," and "emergency warning and communication linkage" into a mobile phone alert system. For example, it corrects the risk level by the movement trajectory of obstacles, verifies the scene by road type and the tilt angle of the mobile phone, and automatically alerts the other party and switches the screen ratio in extremely dangerous situations. This fills the gaps in existing technologies in "dynamic risk adaptation," "accurate scene recognition," and "communication and security collaboration." At the same time, the system expands its functions based on the existing hardware of the mobile phone (camera, GPS, speaker), without the need for additional equipment. It is low-cost, easy to promote, and can quickly cover a wide range of mobile phone users, especially students, office workers, and other groups who frequently "look down at their phones." The system is compatible with mainstream video communication apps and voice call scenarios. The historical data storage module can provide security reports and optimize algorithm parameters, providing long-term and sustainable technical support for the mobile security of "smartphone addicts". Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system flow of the invention. Detailed Implementation

[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0017] In the description of this invention, unless otherwise stated, "a plurality of" means two or more; the terms "upper," "lower," "left," "right," "inner," "outer," "front end," "rear end," "head," "tail," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0018] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0019] A mobile phone video communication and warning system capable of displaying real-time road conditions ahead includes an image acquisition module, a GPS positioning and speed detection module, a data processing module, a road condition display module, a safety warning module, and a video communication adapter module. The modules interact with each other via the mobile phone's internal bus, as detailed below: The image acquisition module includes a wide-angle camera (shooting direction opposite to the screen) and a light compensation unit, which is used to acquire real-time road condition images 1-15m ahead during video communication. The light compensation unit dynamically adjusts the supplementary light according to the ambient light intensity to ensure that the image clarity meets the standard in low light environment. The image acquisition module also includes an obstacle dynamic tracking unit. After the image recognition algorithm detects an obstacle, this unit achieves dynamic tracking through the inter-frame difference method: first, it extracts the pixel coordinates of the obstacle in two consecutive frames of images, and then calculates the pixel displacement. Combined with camera focal length Actual height of the obstacle and the pixel height of obstacles in the image Through formula ( (Real-time updates of obstacle distance, representing the change in distance between the obstacle and the phone) Simultaneously, the obstacle's movement direction is fitted using displacement trajectories from five consecutive frames. (Using the angle directly in front of the phone as 0°, the range is -90° to 90°), and calculate the movement speed. ( (frame interval time). Data processing module receives and Then, an obstacle dynamic correction coefficient is introduced. The risk level formula has been optimized and is now revised as follows: in The rule for determining the value is: when (Move toward the phone) =0.5; (Move away from the phone) =0.1; (Horizontal movement) =0.3; The dynamic tracking unit also sets tracking priorities for different obstacle types: pedestrians > moving vehicles > fixed obstacles. When multiple types of obstacles are detected at the same time, the risk level is calculated based on the high-priority obstacle to ensure more accurate response to dynamic dangerous targets. The GPS positioning and speed detection module obtains location coordinates through the GPS positioning unit, and the inertial measurement unit (IMU) collects acceleration and angular velocity to calculate the movement speed. Based on the speed and rate of change, it determines whether the vehicle is "walking" (0-6 km / h, speed change rate ≤ 0.2 m / s²). 2 () or "electric vehicle driving" (6-30km / h, speed change rate ≤0.5m / s 2 In each scenario, the scenario type is simultaneously sent to the data processing module. The GPS positioning and speed detection module also includes a scene verification unit, which ensures accurate scene type determination through a three-layer verification mechanism: The first layer is basic verification, which uses the speed and rate of change of the IMU to make a preliminary judgment on the scenario. The second layer is environmental verification. It uses GPS positioning data to call the map service API to obtain the road attributes of the current location, including whether it is a non-motorized vehicle lane (width ≥ 1.5m), a pedestrian walkway (non-motorized vehicles are prohibited from passing through), or a mixed lane (non-motorized vehicles and pedestrians are allowed to pass together). If it is initially determined to be "electric vehicle driving" but the road attribute is "pedestrian walkway", then the scenario questioning is triggered. The third layer is for device attitude verification, which uses the phone's gyroscope to collect tilt angles (recorded every 0.5 seconds) and calculates the angle fluctuation value within 10 seconds. (Standard deviation): When riding an electric vehicle, mobile phones are mostly fixed to the handlebars. ≤5°; the handheld state of the mobile phone is unstable when walking. ≥15°; When the second layer of verification triggers a scenario challenge, if the third layer of verification... If the angle is ≥15°, the scene will be corrected to "walking"; if If the angle is ≤5°, the "electric vehicle driving" rule will be maintained, and the map service will be used to further confirm whether it is a "pedestrian crossing where electric vehicles are temporarily allowed" (such as a pedestrian crossing within a residential area). In this case, the scenario coefficient will be adjusted. Adjusted to 1.5 (between walking and electric vehicle scenarios); The data processing module extracts obstacle information (type, straight-line distance from the phone) using a lightweight image recognition algorithm. ), combined with movement speed and scene coefficient (Walking k=1.2, Electric vehicle k=1.8), substitute into the risk level formula ( (Values ​​0-5) Risk Assessment: <1 indicates no risk, only road condition images are transmitted; 1≤ <3 indicates low risk, triggering a low-risk warning and overlaying a yellow-bordered road condition image; A value of ≥3 indicates high risk, triggering a high-risk warning and displaying a road condition image with a red border and obstacle markers. The data processing module also includes an emergency warning unit, which presets extreme risk trigger conditions: in a walking scenario, the distance to an obstacle... ≤1m and moving speed ≥5km / h; in electric vehicle scenarios, ≤3m and ≥15km / h; or obstacle type is "fast-moving vehicle" ( ≥30km / h) and (Approaching the user); When any of the above conditions are met, the emergency warning unit activates the "highest risk" mode: In addition to outputting high-risk warning sounds and vibrations, the safety warning module adds an "intermittent loud sound" (an 85dB sharp warning sound that bursts out every 0.5s and lasts for 2s) and triggers the phone's flashlight to flash (frequency 3Hz), thereby strengthening the warning through multi-sensory stimulation; The video communication adaptation module automatically sends a two-layer prompt to the other party: the first layer is a floating text message "The other party is currently in a dangerous situation and cannot focus on communication at the moment," and the second layer is a 10-second automatic voice broadcast (the content is the same as the text); at the same time, it forcibly switches the communication screen to "road condition priority display" (the road condition screen occupies 1 / 2 of the screen, is displayed in the center, and the communication screen is shrunk to the corner), and pauses the video capture on the user's end (only the road condition screen is transmitted to the other party). When the distance to the obstacle returns to a safe value (walking) ≥2m, electric vehicle After ≥5m and 2s, the system will automatically exit the "highest risk" mode and gradually restore the normal screen ratio and video capture. The communication connection will not be interrupted throughout the entire process. The video communication adaptation module links with the native video communication APP through the API interface to overlay and merge road condition images with communication screens, with a fusion delay of ≤50ms and no interruption of communication. The video communication adaptation module also includes a multi-communication app compatibility unit, which achieves cross-platform compatibility through a general interface and dedicated adaptation. The general interface layer adopts the AccessibilityService framework of Android and the NotificationCenter framework of iOS to monitor the currently active video communication APP process on the mobile phone in real time. When the APP starts a video call, the system's traffic collection and warning functions are automatically triggered. A dedicated adaptation layer is developed for mainstream apps using a dedicated SDK. It directly obtains communication video and audio streams through API interfaces and optimizes video blending efficiency based on the following formula: in, For the total fusion delay, Delay in rendering road condition images Delay for image overlay, This is a weighting coefficient (with a value of 0.6, as rendering accounts for a higher proportion). At the same time, by calling the APP's "picture-in-picture" permission, it is ensured that the traffic image can float on the top layer of the communication interface (highest Z-axis coordinate), without being restricted by the APP's own interface layer; For "voice call" scenarios, the compatibility unit detects the call status through the TelephonyManager interface. When it determines that "a voice call is in progress" and the phone screen is on (brightness ≥ 30 cd / m²), the system will detect the call status. 2 When the traffic condition display and warning function is activated, the traffic condition screen will be adjusted to 1 / 5 of the screen (default lower left corner) to avoid obstructing the dialing interface. The compatibility unit allows users to manually add compatible apps. By selecting a non-preset communication app from the "App List", the system will automatically analyze its interface elements and generate adaptation rules. The traffic display module is based on the mobile phone screen and displays traffic information according to the principle of "communication screen as the main screen and traffic screen as the secondary screen". The transparency of the traffic screen can be adjusted. The traffic display module also includes a gesture and voice control unit, which provides multimodal interaction methods: For gesture control, three gestures and their recognition logic are preset: When two fingers are pinched together, if the distance between the two fingers decreases by ≥30% within 0.5s (calculated based on screen pixel density, such as ≥2cm for a 1080P screen), the traffic image will shrink. Each shrinkage will be 10% of the current size, down to a minimum of 1 / 5 of the screen. Spread two fingers apart. If the distance between the two fingers increases by ≥30% within 0.5s, the road condition image will be magnified. Each magnification is 10% of the current size, up to a maximum of 1 / 3 of the screen. When a single finger touches the screen, the movement trajectory is continuous and the direction is stable (deviation angle ≤ 15°), and the movement distance is ≥ 1cm, the traffic situation screen will be triggered and move with the finger. The movement range is limited to the four corners of the screen (each corner area is 1 / 6 of the screen area). The voice control integrates an offline voice recognition engine (supporting Chinese and English), and the preset command library includes "zoom in on traffic conditions", "zoom out on traffic conditions", "hide traffic conditions", "show traffic conditions", and "switch location". The recognition process is as follows: (1) Collect user voice (via mobile phone microphone, sampling rate 16kHz). (2) Extract speech feature vectors and match them with instruction library templates to calculate the matching degree. ; (3) When When the threshold is adjusted through system settings, execute the corresponding command and provide a "executed" voice prompt. (4) When When the recognition result is displayed, a confirmation pop-up will appear, waiting for the user to click to confirm before execution. The unit also features a mis-touch protection mechanism: when it detects that the user is inputting text (by monitoring the activity status of the input method process), it automatically disables gesture control; when the ambient noise is ≥70dB, it raises the speech recognition threshold to [a higher level]. Reduce false triggers and ensure a balance between interaction efficiency and security; The safety warning module outputs a 1Hz, 60dB warning sound and a 2Hz vibration when the risk is low, and outputs a 2Hz, 75dB warning sound and a 4Hz vibration when the risk is high. The warning sound and communication volume can be adjusted independently. The safety alert module also includes a personalized alert sound unit, which provides multi-dimensional personalized configuration functions: Regarding warning sound types, three basic libraries are preset: natural sound library (including 5 types such as bird calls and stream sounds, suitable for quiet environments), mechanical sound library (including 5 types such as buzzers and pulse sounds, suitable for noisy environments), and voice prompt library (including 10 voices such as "obstacle ahead" and "please slow down", which support custom recording). Users can select "low risk" and "high risk" respectively, and the system will push the appropriate type based on historical usage scenarios (such as recommending mechanical sounds if the user is often in a road environment). The decibel range is set from 50dB to 85dB (in 5dB increments), the vibration intensity is divided into 5 levels (level 1 is slight vibration, level 5 is continuous strong vibration), and it supports a "smart adjustment" mode: real-time collection of ambient noise decibel values ​​via the phone's microphone. ,when When the noise level is ≥60dB, the warning sound will be automatically increased by 10dB and the vibration intensity will be increased by 1 level; when When the vibration level is ≤40dB, it will automatically decrease by 5dB and the vibration intensity will decrease by 1 level. The system also includes a historical data storage module, which adopts a dual-storage architecture of local and cloud storage. The local storage unit records the core data of each risk event, including event time, scenario type, risk level R value, obstacle type and coordinates, and user response behavior (recorded via touch screen operation, such as whether the "attention" button was clicked or whether the speed was reduced). The storage period is 30 days by default, and low-risk events are automatically deleted after the period expires (high-risk and emergency warning events are retained for up to 90 days). Cloud storage (enabled with user authorization) synchronizes local high-risk and emergency warning event data, uses AES-256 encryption for transmission, and supports cross-device synchronization (such as restoring historical data after the user changes their mobile phone). Based on historical data, the system generates a weekly security report, which includes: the number of high-risk events and peak times each week (e.g., 18:00-19:00 is the peak period), the distribution of high-frequency obstacle types (e.g., "shared bicycles" account for 30%), and the average user response time (the interval from the issuance of the warning to the user's action). The data processing module also optimizes scene coefficients using reinforcement learning algorithms. If a user experiences three consecutive "high-risk event failures" (response time > 2 seconds) in a certain scenario, the scenario will be automatically removed from the list. The value increases by 0.2 (maximum value is walking). =1.5, electric vehicle =2.2); If there are 5 consecutive "low-risk event false alarms" (user manually marks "no danger"), then... The value decreased by 0.1 (the lower limit is walking). =1.0, electric vehicle =1.5).

[0020] Case 1: Video Communication Security Protection in Walking Scenarios User A (an office worker) was walking home in the evening (ambient light intensity 30 lux) when he made a video call with his family on WeChat. At this time, the phone screen was fully displaying the communication screen, and the user was focused on the call and did not pay attention to the road conditions ahead. After the system starts up, the wide-angle camera of the image acquisition module captures real-time images of the road ahead, and the light compensation unit automatically activates supplemental lighting (brightness adjusted to 50 lux) to ensure image clarity; the GPS and speed detection modules collect the user's movement speed. =4.2km / h (approximately 1.17m / s), the rate of change of velocity determined by the IMU is ≤0.1m / s. 2 The initial assessment identified the scenario as "walking." The scenario verification unit, using GPS positioning, determined the user was on a "sidewalk," and the phone's gyroscope showed a tilt angle fluctuation of ≤20°. A secondary verification confirmed the scenario as "walking," with a scenario coefficient of [missing value]. =1.2; The data processing module uses the YOLOv8 algorithm to identify a child crossing the pedestrian crossing 5 meters ahead (obstacle type: "pedestrian") and calculates the distance to the obstacle. =5m; simultaneously, the obstacle dynamic tracking unit detects the child moving towards the user (θ=0°), and the speed of movement... =1.5m / s, determine the dynamic correction coefficient. =0.5; Substitute into the revised risk level formula: =1.2×(1.17 / 5)×(1+0.5×(1.5 / 1.17))≈1.2×0.234×(1+0.641)≈1.2×0.234×1.641≈0.46, which is judged as "no risk". The road conditions ahead are only displayed in the upper right corner of the mobile phone screen (1 / 4 screen size, 50% transparency). Children in the picture are marked with a "green box". After 3 seconds, the child suddenly accelerated. =2.5m / s), Shortened to 3m, recalculated =1.2×(1.17 / 3)×(1+0.5×(2.5 / 1.17))≈1.2×0.39×(1+1.077)≈1.2×0.39×2.077≈0.98, close to the "low risk" threshold; another second passed, Further shortened to 2.5m, =1.2×(1.17 / 2.5)×(1+0.5×(2.5 / 1.17))≈1.2×0.468×2.077≈1.15, which is judged as "low risk"; At this moment, the safety warning module activates, emitting a 60dB "beep" warning sound at a frequency of 1Hz, while the phone motor vibrates at a frequency of 2Hz; the traffic display module switches the border of the traffic image to yellow, while maintaining normal display of the communication screen; after sensing the warning, User A sees the child ahead through the traffic image in the upper right corner, immediately slows down and avoids the child, and waits... After the distance was extended to 4m, the radius (R) dropped back to 0.7, the warning stopped, the border of the road condition screen returned to green, communication was not interrupted, and the risk of collision was successfully avoided.

[0021] Case 2: Emergency Warning in Electric Vehicle Driving Scenarios User B (delivery person) was delivering goods by electric bike during the morning rush hour (ambient light intensity 800 lux). He used WeChat to communicate with customers via video to get the delivery address. His phone was fixed to the handlebars of the electric bike with a bracket (tilt angle 10°, stable and without fluctuation). The user occasionally looked down at the communication screen and did not pay full attention to the road conditions ahead. After the system starts up, the GPS and speed detection modules collect the user's movement speed. =20km / h (approximately 5.56m / s), the IMU determines the rate of change of velocity to be ≤0.3m / s. 2 The initial assessment identified the scenario as "electric vehicle driving"; the scenario verification unit confirmed via GPS positioning that the user was located in the "non-motorized vehicle lane," the phone's tilt angle was stable, and secondary verification confirmed the scenario was correct, resulting in a scenario coefficient of [missing information]. =1.8; The wide-angle camera in the image acquisition module captured a shared bicycle that suddenly stopped 8 meters ahead (obstacle type: "fixed obstacle"). The calculation... =8m, obstacle did not move ( =0), dynamic correction coefficient =0.1, substitute into the formula =1.8×(5.56 / 8)×(1+0.1×0)≈1.8×0.695≈1.25, which is judged as "low risk". The safety warning module emits a "beep" warning sound, and the road condition screen displays a yellow border. After hearing the warning, User B slightly reduced his speed to 15 km / h. =4.17m / s), but did not completely avoid it; 2 seconds later, a pedestrian suddenly walked out from next to the shared bicycle in front ( =5m, =1.2m / s, =10°), the data processing module recalculates: =5m, v=4.17m / s, =0.5, =1.8×(4.17 / 5)×(1+0.5×(1.2 / 4.17))≈1.8×0.834×(1+0.144)≈1.8×0.834×1.144≈1.72, still considered "low risk", but Continuing to shorten; 1 second later, The distance is drastically reduced to 2.8m (for pedestrians crossing the non-motorized vehicle lane quickly). =4.17m / s. At this time, the emergency warning unit of the data processing module triggers the "highest risk" mode: on the one hand, the safety warning module switches to a 2Hz, 75dB "beep beep" warning sound, and the motor vibrates strongly at a 4Hz frequency; on the other hand, the video communication adapter module sends a text message to the customer that "currently in a dangerous scene, communication cannot continue for the time being", and switches the communication screen to "road condition priority" (the road condition screen occupies 1 / 2 of the screen, marked "pedestrian ahead, emergency avoidance"); User B immediately braked and slowed down, while simultaneously checking the pedestrian's position on the magnified road view and swerving to the left to avoid them; After expanding to 5 meters, the "highest risk" mode was deactivated, the communication screen returned to normal proportions, and User B continued to communicate with the customer, successfully averting an accident.

[0022] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A mobile phone video communication and warning system capable of displaying real-time road conditions ahead, characterized in that, It includes an image acquisition module, a GPS positioning and speed detection module, a data processing module, a traffic condition display module, a safety warning module, and a video communication adapter module. The modules interact with each other via the phone's internal bus, as detailed below: The image acquisition module includes a wide-angle camera and a light compensation unit, which is used to acquire real-time road condition images 1-15m ahead during video communication. The light compensation unit dynamically adjusts the supplementary light according to the ambient light intensity to ensure that the image clarity meets the standard in low-light environments. The GPS positioning and speed detection module obtains position coordinates through the GPS positioning unit, and the inertial measurement unit collects acceleration and angular velocity to calculate the moving speed. Based on the speed and rate of change, it determines whether the scene is "walking" or "electric vehicle driving" and simultaneously sends the scene type to the data processing module. The data processing module extracts obstacle information using a lightweight image recognition algorithm, combined with movement speed. and scene coefficient Substitute into the risk level formula Assess the risks: <1 indicates no risk, only road condition images are transmitted; 1≤ <3 indicates low risk, triggering a low-risk warning and overlaying a yellow-bordered road condition image; A value of ≥3 indicates high risk, triggering a high-risk warning and displaying a road condition image with a red border and obstacle markers. The video communication adaptation module links with the native video communication APP through the API interface to overlay and merge road condition images with communication screens, with a fusion delay of ≤50ms and no interruption of communication. The traffic display module is based on the mobile phone screen and displays traffic information according to the principle of "communication screen as the main screen and traffic screen as the secondary screen". The transparency of the traffic screen can be adjusted. The safety warning module outputs a 1Hz, 60dB warning sound and a 2Hz vibration when the risk is low, and outputs a 2Hz, 75dB warning sound and a 4Hz vibration when the risk is high. The warning sound and communication volume can be adjusted independently.

2. The system according to claim 1, characterized in that, The image acquisition module also includes an obstacle dynamic tracking unit. After the image recognition algorithm detects an obstacle, this unit achieves dynamic tracking through the inter-frame difference method: first, it extracts the pixel coordinates of the obstacle from two consecutive frames of images, and then calculates the pixel displacement. Combined with camera focal length Actual height of the obstacle and the pixel height of obstacles in the image Through formula ( (Real-time updates of obstacle distance, representing the change in distance between the obstacle and the phone) Simultaneously, the obstacle's movement direction is fitted using displacement trajectories from five consecutive frames. And calculate the movement speed. ( (frame interval time). Data processing module receives and Then, an obstacle dynamic correction coefficient is introduced. The risk level formula has been optimized and is now revised as follows: in The rule for determining the value is: when , =0.5; , =0.1; , =0.3; The dynamic tracking unit also sets tracking priorities for different obstacle types: pedestrians > moving vehicles > fixed obstacles. When multiple types of obstacles are detected at the same time, the risk level is calculated based on the higher priority obstacle to ensure more accurate response to dynamic dangerous targets.

3. The system according to claim 1, characterized in that, The GPS positioning and speed detection module also includes a scene verification unit, which ensures accurate scene type determination through a three-layer verification mechanism: The first layer is basic verification, which uses the speed and rate of change of the IMU to make a preliminary judgment on the scenario. The second layer is environmental verification, which uses GPS positioning data to call the map service API to obtain the road attributes of the current location, including whether it is a non-motorized vehicle lane, a pedestrian lane, or a mixed lane. If it is initially determined to be "electric vehicle driving" but the road attribute is "pedestrian lane", then a scenario questioning is triggered. The third layer is for device attitude verification, which uses the phone's gyroscope to collect tilt angles and calculates the angle fluctuation value within 10 seconds. Because mobile phones are often fixed to the handlebars when an electric vehicle is in motion, ≤5°; the handheld state of the phone is unstable while walking. ≥15°; When the second layer of verification triggers a scenario challenge, if the third layer of verification... If the angle is ≥15°, the scene will be corrected to "walking"; if If the angle is ≤5°, the "electric vehicle driving" rule will be maintained, and the map service will be used to further confirm whether it is a "pedestrian crossing where electric vehicles are temporarily allowed". In this case, the scene coefficient will be adjusted. Adjusted to 1.

5.

4. The system according to claim 1, characterized in that, The safety alert module also includes a personalized alert sound unit, which provides multi-dimensional personalized configuration functions: Regarding the types of warning sounds, three basic libraries are preset: natural sound library, mechanical sound library, and voice prompt library. Users can select "low risk" and "high risk" respectively, and the system will push the appropriate type based on historical usage scenarios. The decibel range is set to 50dB-85dB, the vibration intensity is divided into 5 levels, and it supports a "smart adjustment" mode: it collects the ambient noise decibel value in real time through the mobile phone microphone. ,when When the noise level is ≥60dB, the warning sound will be automatically increased by 10dB and the vibration intensity will be increased by 1 level; when When the vibration level is ≤40dB, it will automatically decrease by 5dB and the vibration intensity will decrease by 1 level.

5. The system according to claim 1, characterized in that, The data processing module also includes an emergency warning unit, which presets extreme risk triggering conditions: in a walking scenario, the distance to an obstacle... ≤1m and moving speed ≥5km / h; in electric vehicle scenarios, ≤3m and ≥15km / h; or the obstacle type is "fast-moving vehicle" and When any of the above conditions are met, the emergency warning unit activates the "highest risk" mode: In addition to outputting high-risk warning sounds and vibrations, the safety warning module adds an "intermittent loud sound" and triggers the phone's flashlight to enhance the warning through multi-sensory stimulation; The video communication adaptation module automatically sends a two-layer prompt to the other party: the first layer is a floating text "The other party is currently in a dangerous situation and cannot focus on communication at the moment," and the second layer is a 10-second automatic voice broadcast; at the same time, it forcibly switches the communication screen to "road condition priority display" and pauses the video capture on the user's end. When the distance to the obstacle returns to a safe value (walking) ≥2m, electric vehicle After ≥5m and 2 seconds, the system will automatically exit the "highest risk" mode and gradually restore the normal screen ratio and video capture. The communication connection will not be interrupted throughout the process.

6. The system according to claim 1, characterized in that, The system also includes a historical data storage module, which adopts a dual storage architecture of local and cloud storage. The local storage unit records the core data of each risk event, including event time, scenario type, risk level R value, obstacle type and coordinates, and user response behavior. The storage period is 30 days by default, and low-risk events are automatically deleted after the period expires. Cloud storage synchronizes local high-risk and emergency warning event data, using AES-256 encrypted transmission and supporting cross-device synchronization; Based on historical data, the system generates a weekly security report, which includes: the number of high-risk events and peak times each week, the distribution of high-frequency obstacle types, and the average user response time. The data processing module also optimizes scene coefficients using reinforcement learning algorithms. If a user fails to respond to a "high-risk event in a timely manner" three times consecutively in a certain scenario, the scenario will be automatically reclassified. The value will increase by 0.2; if there are 5 consecutive "low-risk event false alarms", then... The value decreased by 0.

1.

7. The system according to claim 1, characterized in that, The video communication adaptation module also includes a multi-communication APP compatibility unit, which achieves cross-platform compatibility through a general interface and dedicated adaptation. The general interface layer adopts the AccessibilityService framework of Android and the NotificationCenter framework of iOS to monitor the currently active video communication APP process on the mobile phone in real time. When the APP starts a video call, the system's traffic collection and warning functions are automatically triggered. A dedicated adaptation layer is developed for mainstream apps using a dedicated SDK. It directly obtains communication video and audio streams through API interfaces and optimizes video blending efficiency based on the following formula: in, For the total fusion delay, Delay in rendering road condition images Delay for image overlay, These are the weighting coefficients; At the same time, by calling the APP's "picture-in-picture" permission, it is ensured that the traffic image can float on the top layer of the communication interface, without being restricted by the APP's own interface hierarchy. For the "voice call" scenario, the compatibility unit detects the call status through the telephone status monitoring interface. When it is determined that "a voice call is in progress" and the mobile phone screen is on, the traffic display and warning function is automatically activated. At this time, the traffic display occupies 1 / 5 of the screen to avoid obstructing the dialing interface. The compatibility unit allows users to manually add compatible apps. By selecting a non-preset communication app from the "App List", the system will automatically analyze its interface elements and generate adaptation rules.

8. The system according to claim 1, characterized in that, The traffic display module also includes a gesture and voice control unit, which provides multimodal interaction methods: For gesture control, three gestures and their recognition logic are preset: When two fingers are pinched together, if the distance between the two fingers decreases by ≥30% within 0.5 seconds, the traffic display will shrink, with each shrinkage being 10% of the current size, down to a minimum of 1 / 5 of the screen. Spread two fingers apart. If the distance between the two fingers increases by ≥30% within 0.5s, the road condition image will be magnified. Each magnification is 10% of the current size, up to a maximum of 1 / 3 of the screen. When a single finger touches the screen, if the movement trajectory is continuous and the direction is stable, and the movement distance is ≥1cm, a traffic display will be triggered and the screen will move with the finger. The movement range is limited to the four corners of the screen. The voice control integrates an offline voice recognition engine, with a preset command library including "zoom in on traffic conditions", "zoom out on traffic conditions", "hide traffic conditions", "show traffic conditions", and "switch location". The recognition process is as follows: (1) Collect user voice; (2) Extract speech feature vectors and match them with instruction library templates to calculate the matching degree. ; (3) When When the command is executed, a voice prompt indicating "executed" will be provided. (4) When When the user clicks "confirm," a confirmation pop-up window appears, waiting for the user to click "confirm" before execution. The unit also features a false touch protection mechanism: it automatically disables gesture control when it detects that the user is inputting text; and it raises the voice recognition threshold when the ambient noise level is ≥70dB. This reduces accidental triggering and ensures a balance between interaction efficiency and security.

Images