An intelligent mirror system and method based on cooperation of a mobile terminal and a mirror device

Abstract

The application discloses a kind of intelligent mirror system and method based on mobile terminal and mirror equipment cooperation, and both establish two-way connection by wireless communication.In portable mirror mode, mobile terminal shoots user blind area part, mirror equipment displays picture and receives touch operation, control instruction is sent to mobile terminal to execute image processing;In wall mirror mode, mirror equipment built-in camera collects user image, mobile terminal displays picture and receives touch operation, control instruction is sent to mirror equipment to execute image processing.The application utilizes the high-performance camera of user's existing mobile device, reduces the cost of mirror equipment.The system supports picture zooming, light supplement, photographing, video recording, storage, sharing, voice control, time-lapse photography, picture freeze frame comparison and other functions, adopts modular architecture to support extension, display module is closed during daily use, and no power is consumed.

Description

Technical Field This invention relates to the field of smart home and personal care equipment technology, specifically to a smart mirror system and method that utilizes a mobile terminal (such as a smartphone) and a mirror display device to achieve real-time observation and interactive control of blind spots in the human body. Background Technology Mirrors are indispensable tools in daily life, used for grooming, makeup, shaving, and more. However, traditional mirrors have inherent blind spots, making it difficult for users to directly observe areas such as the back of their head, behind their ears, and their back. This causes inconvenience in situations like hairstyling, skincare, and trying on clothes. In recent years, some smart mirror products have emerged on the market. One type is the "all-in-one smart mirror," which integrates a camera, display screen, and processing unit into the mirror itself, allowing users to capture and display images directly from the mirror. However, these products are relatively expensive, and the camera's resolution and performance are limited by the built-in module, making it difficult to upgrade flexibly with technological advancements. Another existing technology involves simple screen projection between a mobile phone and the mirror, but this typically only achieves one-way image transmission; interactive control still requires the mobile phone. Users cannot operate their phones simultaneously when photographing blind spots, making it inconvenient. Therefore, there is an urgent need for a smart mirror system that can utilize the high-performance shooting capabilities of users' existing mobile devices while also enabling convenient interactive control. Summary of the Invention

[0001] (I) Purpose of the Invention The present invention aims to provide an intelligent mirror system and method based on the collaboration of mobile terminal and mirror device, so as to solve the technical problems in the prior art, such as the difficulty for users to observe their own blind spots, the high cost and difficulty in upgrading existing intelligent mirror products, and the inconvenience of operation.

[0002] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: A smart mirror system and method based on the collaboration of a mobile terminal and a mirror device, comprising: a mobile terminal having a camera, a first wireless communication module, and a first processing unit, for acquiring real-time images of a user-specified part; a mirror device having a mirror body, a second wireless communication module, and a second processing unit, for receiving and displaying the real-time images acquired by the mobile terminal; the mobile terminal and the mirror device establishing a bidirectional wireless communication connection through the first wireless communication module and the second wireless communication module; wherein, the mirror device is further provided with a touch sensing module for receiving touch operations by the user on the mirror body and generating corresponding control commands; the mirror device sending the control commands to the mobile terminal through the bidirectional wireless communication connection; the mobile terminal performing corresponding image processing operations according to the received control commands. Further, the mobile terminal is also provided with a second touch sensing module (i.e., the mobile terminal's own touchscreen), and the user's touch operations on the mobile terminal can also generate control commands and send them to the mirror device, realizing bidirectional interactive control.

[0003] (III) Two working modes First working mode (portable mirror mode): The mobile terminal is used to capture images of blind spots such as the back of the user's head, behind the ears, and back. The mirror device (portable mirror) is used to display the images captured by the mobile terminal in real time. The user performs touch operations on the mirror device, sending control commands to the mobile terminal, which then performs operations such as image zooming, lighting, taking photos, recording videos, storing, and sharing. In this mode, the mobile terminal's camera and the mirror device's display screen are separate. The mobile terminal can be placed or held behind the user, while the mirror device can be held in front of the user for viewing and operation.

[0004] Second working mode (wall mirror mode): The mirror device (wall-mounted or floor-standing) is equipped with a built-in camera to capture full-body or partial images of the user. A mobile terminal is used to display the images captured by the mirror device in real time. The user performs touch operations on the mobile terminal, sending control commands to the mirror device, which then performs operations such as image zooming, lighting, taking photos, recording videos, storing, and sharing. In this mode, the built-in camera of the mirror device is separated from the display screen of the mobile terminal, allowing the user to freely turn around and pose in front of the mirror device while simultaneously viewing and operating the screen on the mobile terminal.

[0005] (iv) Functional Expansion and Modular Design The mirror device and mobile terminal application of this invention adopt a modular design. The mirror device can selectively integrate one or more of the following functional modules according to product positioning and cost requirements: voice control module, ambient light sensor, fill light module, image freeze-frame contrast module, time-lapse shooting module, etc. The combination of these functional modules is not limited to a specific configuration; manufacturers can flexibly combine them according to actual needs to form products with different functional levels.

[0006] The mobile terminal's application adopts a modular architecture. Basic functional modules are pre-installed in the application, while extended functional modules (such as gesture recognition, cloud storage and history, multi-device synchronization, AR virtual makeup try-on, motion-sensing interactive fitness, intelligent beautification, skin analysis, etc.) are optional modules that can be selectively enabled through subsequent software upgrades, dynamic loading, or cloud configuration without requiring replacement of the mobile terminal or mirror device. The scope of protection of this invention is not limited to any specific combination of functions, but covers the technical solution itself for implementing the functions.

[0007] (v) Beneficial effects Compared with the prior art, the present invention has the following beneficial effects: 1. Leveraging the high performance of existing mobile devices: With the help of high-resolution cameras on mobile terminals such as smartphones, the image acquisition quality is far superior to the mid-to-low-end cameras built into the all-in-one smart mirror. At the same time, users can upgrade their mobile terminals at any time as technology advances without having to replace the mirror device.

[0008] 2. Significantly improved ease of operation: In portable mirror mode, users place the mobile terminal behind them to shoot blind spots, while holding the mirror device in front of them to view and operate it; in wall mirror mode, users stand in front of the mirror and move freely, while holding the mobile terminal to view and operate it.

[0009] 3. Low cost and easy to promote: Mirror devices do not require the integration of complex camera modules and high-performance processing units (wall mirror mode requires a built-in camera, but does not require a touch screen), and hardware costs are controllable, which is conducive to the popularization and promotion of the product.

[0010] 4. Feature-rich and flexible interaction: It supports a variety of functions such as screen zoom, fill light, taking photos, recording videos, storage, sharing, voice control, time-lapse shooting, and image freeze-frame comparison. The interaction can be performed on both the mirror screen and mobile devices.

[0011] 5. Zero power consumption in daily use: When the mirror device is used only as a regular mirror, the display screen can be completely turned off, consuming no power and making it environmentally friendly and energy-saving.

[0012] 6. High scalability: It adopts a modular hardware and software architecture, and new functions can be added through software upgrades without replacing hardware, resulting in a long product life cycle. Attached Figure Description

[0013] Figure 1 The system composition and interaction flow diagram of Embodiment 1 of the present invention (portable mirror mode) shows the spatial relationship, wireless connection and data flow between the mobile terminal (phone) and the small mirror device. The diagram is labeled: 101—Mobile terminal (placed behind the user for shooting), 102—Mirror device (held in front of the user for viewing and operation), and a double-headed arrow indicates "video stream + control commands".

[0014] Figure 2 The system composition and interaction flow diagram of Embodiment 2 of the present invention (wall mirror mode) shows the spatial relationship, wireless connection and data flow between a large wall mirror (with built-in camera) and a mobile terminal (phone). The diagram is labeled: 201—Mirror device (user stands in front of the mirror, built-in camera), 202—Mobile terminal (handheld viewing and operation), and a double-headed arrow indicates "video stream + control commands". Detailed Implementation

[0015] Example 1: Portable Smart Makeup Mirror (Portable Mirror Mode) This embodiment provides a portable smart makeup mirror system, including: (1) Mobile terminal: The user's own smartphone is used, which has a rear camera (resolution not limited), a wireless communication module (supporting wireless communication protocols, such as but not limited to Wi-Fi, Bluetooth, etc.), and a touch screen. The smartphone is equipped with a dedicated smart mirroring application (App).

[0016] (2) Mirror device: includes a mirror body with dimensions of approximately 50mm to approximately 150mm (e.g., 75mm, 85mm, 100mm, etc., not limited to specific values), and a shape that can be circular, square, oval, or other suitable. The mirror body has the following features on its rear or side: • Display module: A display module (such as, but not limited to, LED display, OLED display, Micro LED display, QLED display, electronic paper display, or any other display technology capable of displaying images) is attached to the translucent mirror material; • Touch sensing module: A touch sensing module (such as, but not limited to, capacitive touch screen, resistive touch screen, optical touch screen, etc.) is used to cover the display area; • Wireless communication module: Supports wireless communication protocols (such as, but not limited to, Wi-Fi, Bluetooth, etc.); • Processing unit: A processing unit (such as an embedded processor) is used to drive the display, receive touch signals, and communicate with the mobile terminal; • Power module: Rechargeable lithium battery; • Optionally, the microphone module: employs a voice recognition module (e.g., a low-power offline voice recognition module) to receive user voice commands (such as "take a photo", "record a video", "zoom in", "share", etc.) to achieve voice control functions; • Optionally, an ambient light sensor is used to detect the intensity of ambient light and work with the supplementary lighting module to achieve adaptive supplementary lighting; • Mirror body: Using one-way transparent glass or mirror film bonding technology, when the display module is off, the mirror body presents a conventional reflective mirror effect; when the display module is on, the mirror body transmits the light from the display module while maintaining a certain reflective effect.

[0017] (3) Work process: Users open the Smart Mirror App on their smartphones and select "Portable Mirror Mode." The App activates the phone's rear camera and establishes a connection with the mirror device via wireless communication.

[0018] Users place or hold their smartphones behind their backs, above their heads, or in other blind spots where they need to observe. The phone's camera captures images of that area in real time. The image data is transmitted wirelessly to the mirror device, which displays the image on a touchscreen in real time (optionally mirrored).

[0019] Users view live footage on the mirror device and perform touch operations on the mirror surface: pinching two fingers to zoom in and out, tapping the "Take Photo" button to save the current image, tapping the "Record Video" button to record video, and sliding the "Fill Light" slider to adjust the phone's flash brightness, etc. The mirror device transmits the user's commands wirelessly to the smartphone, which performs the corresponding image processing operations and then resends the processed image back to the mirror device for display.

[0020] Users can also operate the device via voice commands: simply say "take a picture" into the microphone of the mirror device, and the device will recognize the command and send it to the phone to take the picture. Voice control is especially convenient when the user's hands are occupied.

[0021] When a user uses the "Freeze Frame Comparison" function, the system will display the newly taken photo and the live screen simultaneously in a split-screen or picture-in-picture manner after taking the photo, making it convenient for the user to compare the differences before and after (such as makeup effects, hairstyle adjustments, etc.).

[0022] When using the "time-lapse shooting" function (which can be used in both modes, covering photo and / or video recording), users can set a countdown (e.g., 3 seconds, 5 seconds, 10 seconds) on the mirror device or mobile app. After pressing the shutter button, the system will automatically count down and perform the shooting operation. If time-lapse video recording is selected, the recording duration can be set at the same time (e.g., 15 seconds, 30 seconds, 1 minute). The recording will start automatically after the countdown ends and stop automatically after the set duration is reached.

[0023] When users do not need to use the smart functions, they can turn off the display module of the mirror device, and the mirror device will revert to a normal portable mirror without consuming power.

[0024] Example 2: Home-style smart wall mirror (wall mirror mode) This embodiment provides a home-use smart wall mirror system, including: (1) Mirror device: A mirror device installed on a wall, tabletop, or floor, with a mirror body size of approximately 10 inches or larger (e.g., 12 inches, 13 inches, 15 inches, 21 inches, 32 inches, etc., not limited to specific values). Its components include: • Mirror body: Utilizes one-way transparent glass or mirror film bonding technology; • Display module: A display module (such as, but not limited to, LED display, OLED display, Micro LED display, QLED display, or any other display technology capable of displaying images) is attached to the translucent mirror material; • Built-in camera: Installed on the top, bottom or side frame of the mirror body, facing the area in front of the mirror, to capture full-body or partial images of the user (camera resolution is not limited). • Wireless communication module: Supports wireless communication protocols (such as, but not limited to, Wi-Fi) for video streaming and command reception; • Supplemental lighting module: Optionally, an LED light strip can be installed around the mirror body to supplement lighting in low-light environments; • Processing unit: A processing unit (such as an embedded processor) is used to control camera acquisition, video encoding, wireless transmission, and command response; • Power module: Connects to AC power; • Optionally, a microphone module: used to receive the user's voice commands; • Optionally, an ambient light sensor is used to detect the intensity of ambient light and automatically adjust the brightness or color temperature of the supplementary lighting module.

[0025] (2) Mobile terminal: The user’s own smartphone is used and a dedicated smart mirror application (App) is installed.

[0026] (3) Work process: Users open the Smart Mirror App on their smartphones and select "Wall Mirror Mode." The App establishes a connection with the wall mirror device via wireless communication, and the wall mirror's built-in camera begins to capture real-time images of the area in front of the mirror.

[0027] Users can stand in front of the wall mirror and freely turn around, pose, try on clothes, etc. The wall mirror device streams real-time images captured by the camera to a smartphone via wireless communication, and the smartphone app displays the images.

[0028] Users view the live feed on their smartphones and perform touch operations on the screen: pinch to zoom in and out to view details of the back, tap "Take Photo" to save a full-body or partial photo, tap "Record Video" to record a video, and slide to adjust the brightness of the wall mirror's fill light, etc. The mobile app transmits the user's commands wirelessly to the wall mirror device, which executes the corresponding operations and can then push the processed image back to the phone for display.

[0029] Voice control (preferred option): In scenarios where hands are inconvenient to touch the screen, such as when trying on clothes or posing, users can directly control the system via voice commands. This is the most convenient operation method in this mode. Users can say "Start recording" into their phone's microphone or the wall mirror's microphone to start recording video; say "Stop recording" to stop recording and save the video; say "Take a photo" to immediately take a picture; and say "Zoom in" or "Zoom out" to adjust the screen zoom accordingly. Voice control eliminates the need for users to touch any buttons, completely freeing their hands and greatly improving ease of use.

[0030] Time-lapse photography (supplementary option): Users can also use the time-lapse photography function. On the mobile app, select "Time-lapse photo" or "Time-lapse video," set the countdown (e.g., 3 seconds, 5 seconds, 10 seconds) and the recording duration (if applicable). After pressing the shutter button, the phone screen displays the countdown numbers. The user can then place the phone down or hold it behind their back and pose comfortably. After the countdown ends, the system automatically takes a photo or starts recording video. The time-lapse photography function is suitable for scenarios where users want to preset a countdown and prepare their poses at their own pace.

[0031] Users can use the freeze-frame comparison function: after taking a photo, the phone screen will display the photo taken and the real-time image in a split screen, making it easy to compare the effect of clothing, adjust posture, etc.

[0032] Users can store the photos or videos they take in their phone's album, or share them with one click through the app to social media platforms such as WeChat and email, or send them to other smart devices (such as computers and tablets).

[0033] Example 3: Functional Design of Mobile Terminal App The smart mirror app of this invention runs on a mobile terminal, adopts a modular architecture, and has the following core functional modules: • Mode selection module: Based on the user's selected device type or automatic detection, switch to "Portable Mirror Mode" (phone capture → mirror display) or "Wall Mirror Mode" (mirror capture → phone display); • Video capture and encoding module: In portable mirror mode, the mobile phone camera is used to capture video, which is then encoded, compressed, and transmitted to the mirror device in real time via wireless communication; • Video decoding and rendering module: In wall mirror mode, it receives the video stream pushed by the mirror device, decodes it, renders it, and displays it on the mobile phone screen; • Command transceiver module: Communicates bidirectionally with the mirror device via a wireless communication protocol to send and receive operation commands; • Image processing module: Responds to operation commands such as zooming, taking pictures, recording videos, adding light, time-lapse shooting (including taking pictures and recording videos), and freeze-frame comparison, and calls the corresponding functions to perform image processing; • Storage and sharing module: Store captured images or videos in the phone's local album, or send them to other applications or devices via the system's sharing function; • Extended Function Interface: Reserved interface for future software upgrades to add extended functions such as gesture recognition, cloud storage and history, multi-device synchronization, AR virtual makeup try-on, motion-sensing interactive fitness, intelligent beautification, and skin analysis, without requiring hardware replacement.

[0034] Example 4: Technical Implementation of Low-Latency Video Transmission To achieve a smooth user experience, This invention preferably employs a low-latency wireless transmission method. This invention does not limit the specific wireless transmission protocol; any wireless communication method capable of enabling real-time video streaming can be used.

[0035] As an example and not a limitation, the video transmission may be implemented using one or more of the following methods: Wi-Fi Direct, Wi-Fi LAN transmission, Bluetooth transmission, UWB transmission, 4G / 5G cellular network, WebRTC protocol, RTSP protocol, RTMP protocol, or a custom UDP proprietary transport protocol. In a LAN environment, using the WebRTC protocol in conjunction with adaptive bit rate adjustment and forward error correction (FEC) technology can control the end-to-end latency to a low level (e.g., within 100ms), thereby achieving a smooth user experience.

[0036] Example 5: Miniaturized Camera Selection Scheme (Applicable to Wall Mirror Mode) In wall mirror mode, to maintain the aesthetics and concealment of the mirror, a miniaturized camera module is recommended. For example, a surface-mount miniature image sensor (with a size as small as 1mm × 1mm) can be used, suitable for integration within the mirror frame, achieving a near-invisible integration effect. Higher resolution camera modules (e.g., but not limited to the megapixel level) can also be selected to provide clearer image quality.

Claims

1. Claim 1: A smart mirroring system based on the collaboration of a mobile terminal and a mirror device, characterized in that, include: • A mobile terminal, comprising a camera, a first wireless communication module, and a first processing unit, for acquiring real-time images of a user-specified body part; • A mirror device, comprising a mirror body, a second wireless communication module, and a second processing unit, for receiving and displaying data collected by the mobile terminal. Real-time images; The mobile terminal and the mirror device establish bidirectional wireless communication through the first wireless communication module and the second wireless communication module. connect; The mirror device is also equipped with a touch sensing module for receiving touch operations from the user on the mirror body and generating corresponding control signals. Control instructions; The mirror device sends the control commands to the mobile terminal via the bidirectional wireless communication connection; the mobile terminal... Based on the received control commands, execute the corresponding image processing operations.

2. Claim 2: The intelligent mirroring system according to claim 1, characterized in that, The image processing operations include at least one of the following: image zooming, image magnification, image reduction, adjusting camera fill light parameters, taking a picture, recording a video, image storage, image sharing, and image freeze-frame comparison.

3. Claim 3: The intelligent mirroring system according to claim 1, characterized in that, The mirror device is a portable mirror device with a mirror body size of approximately 50mm to approximately 150mm; the camera of the mobile terminal is used to capture images of the back of the user's head, behind the ears, or the back of the body, and the mirror device is used to display the images captured by the mobile terminal in real time.

4. Claim 4: The intelligent mirroring system according to claim 1, characterized in that, The mirror device is a wall-mounted, desktop, or floor-standing mirror device, and it is also equipped with a built-in camera for capturing full-body or partial images of the user; the mobile terminal is used to display the images captured by the mirror device in real time; the touch sensing module of the mobile terminal is used to receive user operations, generate control commands and send them to the mirror device; the size of the mirror device is approximately 10 inches or larger.

5. Claim 5: The intelligent mirroring system according to claim 1, characterized in that, The mirror device also includes a microphone module for receiving user voice commands.

6. Claim 6: The intelligent mirroring system according to claim 1, characterized in that, The mirror device or the mobile terminal also includes an ambient light sensor and a fill light module, used to detect the intensity of ambient light and automatically adjust the brightness or color temperature of the fill light.

7. Claim 7: The intelligent mirroring system according to claim 1, characterized in that, The system supports time-lapse shooting, where users can set the countdown duration. After pressing the shutter button, the system automatically counts down and performs the shooting operation. The shooting operation includes taking photos and / or recording videos.

8. Claim 8: The intelligent mirroring system according to claim 1, characterized in that, The system supports a freeze-frame comparison function, which displays the captured image and the real-time image simultaneously in a split-screen or picture-in-picture manner after a photo is taken.

9. Claim 9: The intelligent mirroring system according to claim 1, characterized in that, The mobile terminal's application adopts a modular architecture, in which basic functional modules are pre-installed in the application, and extended functional modules can be added during subsequent use through network download, dynamic loading, or cloud configuration, without the need to replace the mobile terminal or mirror device.

10. Claim 10: A smart mirroring method based on the collaboration of a mobile terminal and a mirror device, characterized in that, Includes the following steps: • Establish a two-way wireless communication connection between the mobile terminal and the mirror device; • Use the camera on the mobile terminal to capture real-time images of the user's designated area and send them to the mirror device for display via wireless communication; • The touch sensor module of the mirror device receives the user's touch operation, generates control commands, and sends them to the mobile terminal via wireless communication; • The mobile terminal executes corresponding image processing operations based on the received control commands and then displays the processed image on the mirror device. Optionally, the built-in camera of the mirror device can be used to capture full-body or partial images of the user, which can then be transmitted wirelessly to a mobile terminal for display. The touch sensing module of the mobile terminal is used to receive user operations, generate control commands, and send them to the mirror device to perform image processing.

Images