Offline voice interaction monitoring device and system

Abstract

The utility model discloses an off-line voice interaction monitoring device and system, and the device comprises a holder, a communication unit, an energy storage unit, an audio unit, an off-line voice recognition module, and a control unit. The holder is integrated with a camera and has a preset rotation angle; the communication unit is used for performing remote communication with a back-end platform; the energy storage unit is used for supplying power to equipment; the audio unit comprises a loudspeaker and a pickup microphone; the loudspeaker is used for voice broadcasting; the pickup microphone is used for acquiring field sound; and the off-line voice recognition module is used for responding to sound acquired on site according to a preset off-line voice instruction. The technical scheme of the utility model breaks through the dependence of the traditional scheme on network and wired operation, improves the operation convenience and reliability in a complex scene, avoids the potential safety hazard caused by data transmission, and is suitable for scenes with high requirements on real-time performance and autonomy, such as electric power inspection, emergency rescue, field operation and the like.

Description

Technical Field

[0001] This utility model relates to the technical field of mobile monitoring equipment in power system operation and maintenance and emergency repair, and in particular to an offline voice interactive monitoring device and system. Background Technology

[0002] In power system operation and maintenance, emergency repair and on-site operation scenarios, 5G surveillance cameras, as mobile monitoring devices that integrate high-definition video acquisition, wireless transmission and intelligent analysis, have become an important tool for improving operation safety and efficiency.

[0003] Traditional control balls mainly rely on wired operation or remote wireless commands to control the equipment. However, power operation sites (such as substations and field transmission lines) often have strong electromagnetic interference, dust pollution or narrow spaces, which makes traditional button operation and touch screen control susceptible to the influence of the physical environment, resulting in response delay or accidental touch risk.

[0004] For example, maintenance personnel wearing insulated gloves may find it difficult to accurately operate the touchscreen, and the device buttons are prone to malfunction in high temperature and humidity environments. Existing 5G surveillance cameras mostly rely on the network for remote command interaction, but if the network signal is weak on-site (such as in underground cable tunnels or mountainous base station coverage blind spots), or if communication infrastructure is damaged in emergency scenarios, the remote control function may completely fail, causing the device to be unable to adjust the monitoring angle, start or stop recording, and other critical operations in a timely manner, affecting on-site situational awareness.

[0005] For example, during power emergency repairs, maintenance personnel need to simultaneously handle equipment debugging, fault diagnosis, and data recording. Traditional manual control of the PTZ camera requires additional manpower, which can easily lead to operational delays, especially in emergency scenarios where personnel are scarce. Furthermore, occasional equipment failures may result in the inability to connect to the network on-site, causing limited operational failures. Utility Model Content

[0006] To overcome the shortcomings of existing technologies, an offline voice interactive monitoring device and system is proposed. This breaks through the dependence of traditional solutions on network and wired operation, improves the ease of operation and reliability in complex scenarios, and avoids the security risks caused by data leakage. It is suitable for scenarios with high requirements for real-time performance and autonomy, such as power inspection, emergency rescue, and field operations.

[0007] An offline voice-interactive monitoring device, comprising:

[0008] A pan-tilt unit, having a preset rotation angle, and an integrated camera; the camera is used to acquire image information of the deployment site of the offline voice interaction monitoring device.

[0009] The communication unit is used for remote communication with the backend platform;

[0010] An energy storage unit is used to power the offline voice interaction monitoring device;

[0011] An audio unit, comprising a speaker and a microphone; the speaker is used to broadcast voice messages to the deployment site of the offline voice interaction monitoring device; the microphone is used to acquire sound from the deployment site of the offline voice interaction monitoring device.

[0012] An offline speech recognition module is used to respond to the sound acquired by the microphone according to a pre-set offline speech command;

[0013] The control unit is used to control the gimbal, camera, communication unit, energy storage unit, and audio unit.

[0014] Preferably, it also includes a display unit controlled by the control unit for displaying the status information of the offline voice interaction monitoring device.

[0015] Preferably, the energy storage unit includes a lithium battery and a power charging and discharging management module electrically connected thereto.

[0016] Preferably, it also includes an HDMI output unit controlled by the control unit, the HDMI output unit being used to provide HDMI signals to the outside.

[0017] Preferably, the communication unit includes a 5G communication module and an AI chip, and has built-in GPS, BeiDou positioning and Bluetooth functions.

[0018] Preferably, the gimbal's rotation angle includes 360 degrees horizontally and ±90 degrees vertically.

[0019] Preferably, the display unit includes an LCD screen, which is connected to the control unit via I2C.

[0020] Preferably, the HDMI output unit is an external HDMI interface integrated into the control unit.

[0021] Preferably, the camera is connected to the control unit via Ethernet; the pan-tilt unit is connected to the control unit via a 485 bus; the offline voice recognition module is connected to the control unit via a UART interface; and the communication unit is connected to the control unit via UART, USB, and MIPI.

[0022] An offline voice interaction monitoring system includes multiple offline voice interaction monitoring devices as described above, and a backend platform that communicates remotely with the offline voice interaction monitoring devices; the offline voice interaction monitoring devices are deployed at a preset site; the backend platform includes at least a cloud server.

[0023] This utility model provides an offline voice-interactive monitoring device, which, as a front-end, integrates an offline voice recognition module. It supports voice control of pan-tilt-zoom (PTZ) angle rotation, recording start / stop, power sleep mode, AI algorithm control, device self-test, and device status reading. This enables independent voice command parsing and device control even in network-free environments, improving operational convenience and reliability in complex scenarios. Consequently, it eliminates the need for dedicated personnel to operate the device. Within a 5-10 meter range, key operations such as adjusting the monitoring angle, starting / stopping recording, power sleep mode, and AI algorithm control can be performed via offline voice commands. Through the PTZ and camera, it can autonomously monitor facial recognition, safety helmets, work clothes, masks, insulating gloves, and insulating boots. This frees up manpower while enhancing the device's operability in environments with strong electromagnetic interference, dust pollution, confined spaces, underground cable tunnels, and mountainous base station coverage blind spots. It avoids situations where the device cannot complete operations and inspections due to lack of network or damage to wired operating components after arriving on-site, thus improving the safety and efficiency of power system operation and maintenance, emergency repairs, and on-site operations.

[0024] Meanwhile, the system based on this offline voice interactive monitoring equipment, including the monitoring platform including the cloud server, can collect on-site images and video information in real time when connected, and provide guidance or communication to on-site personnel through the audio unit, thereby improving work efficiency in scenarios such as power operation and maintenance and emergency repair. Attached Figure Description

[0025] Figure 1 This is a structural framework diagram of the offline voice interaction monitoring system in this embodiment of the present invention;

[0026] Figure 2 This is a topology diagram of the offline voice interaction monitoring system in this embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram illustrating the working principle of the offline voice interaction monitoring system in this embodiment of the present invention. Detailed Implementation

[0028] To make the technical problems, technical solutions, and beneficial effects of this utility model clearer, the present utility model will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present utility model and are not intended to limit the present utility model.

[0029] Provide an offline voice-interactive monitoring device and system, such as Figures 1 to 3 As shown, the offline voice interactive monitoring system includes multiple offline voice interactive monitoring devices and a backend platform that communicates remotely with these offline voice interactive monitoring devices; the offline voice interactive monitoring devices are deployed at a preset site, such as around power lines; the backend platform includes at least a cloud server, which is a platform used to monitor the site conditions.

[0030] Offline voice-interactive monitoring equipment includes:

[0031] The pan-tilt unit has a preset rotation angle and an integrated camera; the camera is used to acquire image information from the deployment site of the offline voice interaction monitoring equipment.

[0032] Specifically, the camera uses a 20x optical zoom 2MP sensor, capable of clearly capturing target details within a distance of 80 to 200 meters, significantly reducing equipment deployment density. A single device can meet different depth-of-field requirements through zoom adjustment. The gimbal can rotate 360° horizontally and ±90° vertically, and in conjunction with the zoom camera and intelligent algorithms, it can maintain continuous zoom tracking of personnel on site, achieving millisecond-level recognition of faces, safety helmets, work clothes, etc., within 200 meters.

[0033] The communication unit is used for remote communication with the backend platform.

[0034] Specifically, a 5G communication module can be used to provide a low-latency upload channel for data transmission, supporting adaptive switching between 5G millimeter wave and Sub-6GHz dual frequency bands, which greatly reduces packet loss rate in complex environments.

[0035] Energy storage unit is used to power offline voice interaction monitoring equipment.

[0036] Specifically, the energy storage unit uses a large-capacity lithium battery, which supports the device to work continuously for more than 14 hours, ensuring that the device can operate stably online around the clock in multiple scenarios; avoiding situations where the power is insufficient to complete the inspection task or where an additional battery needs to be carried.

[0037] The audio unit includes a speaker and a microphone; the speaker is used to broadcast voice messages to the deployment site of the offline voice interactive monitoring equipment; the microphone is used to acquire sound from the deployment site of the offline voice interactive monitoring equipment.

[0038] Specifically, the audio unit includes a 3W speaker and a long-range microphone. Combined with software functionality, it supports real-time voice communication, providing strong support for remote real-time command. The speaker power and microphone pickup capability allow back-end personnel to accurately direct staff working within a 10-meter range of the monitoring equipment.

[0039] The offline speech recognition module is used to respond to the sound acquired by the microphone according to the pre-set offline speech commands.

[0040] Specifically, the offline voice recognition module uses an independent chip to support voice control of gimbal rotation angle, start / stop recording, power sleep mode, AI algorithm control, device self-test, device status reading, and other operations. It enables independent voice command parsing and device control in environments without network access, improving the ease of operation and reliability in complex scenarios.

[0041] The control unit is used to control the pan-tilt unit, camera, communication unit, energy storage unit, and audio unit.

[0042] Specifically, the control unit includes an MCU, which is responsible for controlling the power supply of each unit, acquiring data, and communicating with the communication unit.

[0043] The working principle of this offline voice interactive monitoring device is as follows:

[0044] By utilizing the voice command recognition and external serial communication functions of the offline voice recognition module, voice commands for operations such as pan-tilt rotation, start / stop recording, power sleep, AI algorithm control, device self-test, and reading device power information and network status are pre-set. When the offline voice recognition module detects the pre-set offline voice command, it communicates with the device's motherboard MCU through the serial port, thereby enabling the motherboard to execute the corresponding operation. This achieves independent voice command parsing and device control in a network-free environment, breaking the device's dependence on local wired operation in network-free situations and improving the ease of operation and reliability in complex scenarios.

[0045] The workflow of this offline voice interactive monitoring device is as follows:

[0046] By placing the device in a location that can monitor the entire power site work area and powering it on, all power supplies to the device are automatically turned on by default. On-site personnel within 5 meters of the device can operate it step-by-step using pre-set offline voice commands, such as checking device status, activating facial recognition and safety helmet recognition, and rotating the pan-tilt unit left and right. Upon receiving the command, the offline voice recognition module transmits the information to the mainboard MCU via serial port. The mainboard MCU then executes the corresponding operation based on the command information and the 5G communication module, and controls the audio unit's speaker to broadcast the execution result.

[0047] In the event of low power, some power sources can be turned off via offline voice commands to reduce power consumption, such as turning off HDMI output or network connection. This enables the standardized and safe process of power system operation and maintenance, emergency repair, and on-site operations, and promotes the construction of smart construction sites.

[0048] Furthermore, the energy storage unit includes a lithium battery and a power charging and discharging management module electrically connected to it.

[0049] Furthermore, it also includes a display unit controlled by the control unit, used to display the status information of the offline voice interactive monitoring device. The display unit includes an LCD screen, which communicates with the control unit via I2C.

[0050] Furthermore, it also includes an HDMI output unit controlled by the control unit, which is used to provide HDMI signals to external devices. The HDMI output unit is an external HDMI interface integrated into the control unit.

[0051] Furthermore, the camera is connected to the control unit via Ethernet; the pan-tilt unit is connected to the control unit via a 485 bus; the offline voice recognition module is connected to the control unit via a UART interface; and the communication unit is connected to the control unit via UART, USB, and MIPI.

[0052] In a preferred implementation, the offline voice interaction monitoring device includes the following units:

[0053] Energy storage unit: The energy storage unit consists of a high-capacity 12V lithium battery and a charging and discharging circuit, including overcharge and over-discharge protection, and is compatible with 18V DC adapter charging.

[0054] Zoom camera unit: The zoom camera unit consists of a 20x optical zoom 200W pixel network camera, which supports local direct connection and standard protocols such as GB / T28181 and ONVIF. It can clearly capture target details within a distance of 80 meters to 200 meters, meeting the requirements of the power platform.

[0055] Gimbal unit: The gimbal unit works with the camera and supports 360° horizontal rotation and ±90° vertical rotation. The horizontal speed is 0-90° / s and the vertical speed is 10° / s, which can quickly track the movement of people.

[0056] 5G Communication Unit: The 5G communication unit mainly consists of a 5G communication module and an AI chip. It has built-in GPS, Beidou positioning and Bluetooth 5.0 functions. It is mainly responsible for communicating with the platform, running AI algorithms, performing face recognition, and recognizing the wearing of work clothes, insulating gloves and insulating boots. It also supports uploading H.264 and H.265 video formats to the platform.

[0057] Audio Unit: The audio unit consists of a 3W speaker and a long-range microphone, supporting real-time remote voice communication and providing strong support for remote real-time command.

[0058] Control Unit: The control unit consists of a control MCU and an HDMI output unit. The control MCU is responsible for controlling the power supply of each unit, data acquisition, and communication with the 5G communication unit. The HDMI output unit converts the MIPIDSI signal integrated into the 5G communication module into an HDMI signal and outputs it to an external display or portable recording device, supporting video conversion and transmission up to 1080@60Hz. Combined with a display, it can meet emergency command needs in situations without network access. Simultaneously, network transmission and HDMI output work in parallel to ensure that at least one data output channel is available in extreme circumstances.

[0059] Display Unit: The display unit consists of an LCD screen, which mainly displays power information and network status. It allows staff to view the basic power level and network status of the equipment in real time, avoiding bringing low power to the site. It can also help determine the network situation on site.

[0060] Offline speech recognition module: The offline speech recognition module uses a local speech trigger engine recognition chip and the latest neural network algorithm. It has the advantages of accurate recognition and low false judgment rate. It can reliably recognize in the far field of 5 meters and supports voice control of pan-tilt rotation angle, start and stop recording, power sleep mode, AI algorithm control, device self-test, device power information and network status reading and other operations.

[0061] In this implementation, the offline voice interaction monitoring device operates as follows:

[0062] The energy storage unit includes an energy storage lithium battery and a charging and discharging circuit to provide power to the device. After the device is powered on, the display unit shows the current power level and network status. The device automatically turns on the camera power and activates the microphone function of the audio unit to record audio. The zoom camera unit records real-time video of the scene and saves it locally. The recorded footage is processed by AI algorithms through the 5G communication unit, thereby controlling the gimbal to continuously zoom and track the on-site personnel, as well as perform basic face recognition and identification of the correct wearing of safety helmets, insulating gloves, and insulating boots.

[0063] When a face matching failure occurs, or when it is detected that safety helmets, insulating gloves, or insulating boots are not worn correctly, the error will be marked on the screen and reported to the platform. The real-time recorded footage is uploaded to the platform via a 5G network. Back-end personnel can use the platform's voice intercom function to control the audio unit of the surveillance camera to speak to on-site staff, who can directly respond to the platform's messages within a 10-meter range.

[0064] When there is no network connection at the site, on-site personnel can connect an external recorder or monitor with an HDMI interface via the HDMI port on the control unit to view the detailed environment of the work area. Alternatively, they can connect a laptop to the device's local network port to access the camera preview interface and operate the PTZ camera to view the work environment and personnel status. When there is no network and manpower is limited, or the local wired operation is malfunctioning and cannot be operated, or the LCD display is damaged and cannot display the device's battery and network status, offline voice commands can be used to control the PTZ camera's rotation angle, start and stop recording, power-on sleep mode, AI algorithm control, device self-test, and read device battery information and network status.

[0065] The above is a description of the present utility model to help understand it; however, the implementation of the present utility model is not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the principle of the present utility model shall be considered equivalent substitutions and shall be included within the protection scope of the present utility model.

Claims

1. An offline voice-interactive monitoring device, characterized in that, include: A gimbal, wherein the gimbal has a preset rotation angle and a camera is integrated on the gimbal; The camera is used to acquire image information of the deployment site of the offline voice interaction monitoring device; The communication unit is used for remote communication with the backend platform; An energy storage unit is used to power the offline voice interaction monitoring device; An audio unit, comprising a speaker and a microphone; the speaker is used to broadcast voice messages to the deployment site of the offline voice interaction monitoring equipment. The microphone is used to acquire sound from the deployment site of the offline voice interaction monitoring device. An offline speech recognition module is used to respond to the sound acquired by the microphone according to a pre-set offline speech command; The control unit is used to control the gimbal, camera, communication unit, energy storage unit, and audio unit.

2. The offline voice interaction monitoring device as described in claim 1, characterized in that, It also includes a display unit controlled by the control unit, used to display the status information of the offline voice interaction monitoring device.

3. The offline voice interaction monitoring device as described in claim 1, characterized in that, The energy storage unit includes a lithium battery and a power charging and discharging management module electrically connected to it.

4. The offline voice interaction monitoring device as described in claim 1, characterized in that, It also includes an HDMI output unit controlled by the control unit, the HDMI output unit being used to provide HDMI signals to the outside.

5. The offline voice interaction monitoring device as described in claim 1, characterized in that, The communication unit includes a 5G communication module and an AI chip, and has built-in GPS, BeiDou positioning and Bluetooth functions.

6. The offline voice interaction monitoring device as described in claim 1, characterized in that, The gimbal's rotation angle includes 360 degrees horizontally and ±90 degrees vertically.

7. The offline voice interaction monitoring device as described in claim 2, characterized in that, The display unit includes an LCD screen, which is connected to the control unit via I2C.

8. The offline voice interaction monitoring device as described in claim 4, characterized in that, The HDMI output unit is an external HDMI interface integrated into the control unit.

9. The offline voice interaction monitoring device as described in any one of claims 1 to 6, characterized in that, The camera is connected to the control unit via Ethernet; the pan-tilt unit is connected to the control unit via a 485 bus; the offline voice recognition module is connected to the control unit via a UART interface; and the communication unit is connected to the control unit via UART, USB, and MIPI.

10. An offline voice-interactive monitoring system, characterized in that, It includes multiple offline voice interaction monitoring devices as described in any one of claims 1 to 9, and a backend platform that communicates remotely with the offline voice interaction monitoring devices; the offline voice interaction monitoring devices are deployed at a preset site; The backend platform includes at least a cloud server.

Images