Multimode-based multimedia equipment automatic inspection system and method

Abstract

The invention provides a multi-mode-based automatic inspection system and method for multimedia equipment, and the system comprises a data collection layer which is used for obtaining multi-mode data and operation logs of the multimedia equipment; the intelligent analysis layer is used for generating an equipment health index and predicting the fault probability of the multimedia equipment; and the dynamic scheduling layer is used for carrying out routing inspection path planning and priority queue management on the multimedia equipment. According to the technical scheme, the problem identification comprehensiveness in the automatic inspection process of the multimedia equipment is improved.

Description

Background Art

[0002] The existing multimedia equipment inspection system has the following defects: 1. Low efficiency of manual inspection: Existing technologies rely on regular manual inspections, which can easily miss hidden problems (such as circuit aging and software compatibility failures).

[0003] 2. Static inspection plan: Traditional solutions use fixed-period inspections (such as once a day), which cannot be dynamically adjusted according to the actual frequency of equipment use, resulting in a waste of resources.

[0004] 3. Single-dimensional diagnosis: Faults are judged only by device status indicators or simple self-test procedures, and complex problems (such as audio distortion and video frame loss) cannot be identified.

[0005] 4. Lack of predictive maintenance: Potential failures cannot be predicted in advance, resulting in sudden equipment downtime. Summary of the invention

[0006] The present application provides a multi-modal multimedia equipment automatic inspection system and method to improve the comprehensiveness of problem identification.

[0007] In a first aspect, a multi-modal multimedia device automatic inspection system is provided, comprising: Data collection layer, used to obtain multimodal data and operation logs of multimedia devices; An intelligent analysis layer, for generating a device health index and predicting a failure probability of the multimedia device; The dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia devices.

[0008] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0009] In a specific implementation scheme, it also includes: an execution and feedback layer, wherein: The execution and feedback layer is used to repair the multimedia device and generate a visual report.

[0010] In a specific implementation scheme, the data collection layer includes: Multimodal sensors, including an integrated audio acquisition module, video quality analysis unit and hardware sensors; Device log interface, used to obtain device operation logs in real time.

[0011] In a specific implementation scheme, the intelligent analysis layer includes: Multimodal data fusion engine, which is used to integrate audio, video, and sensor data to generate device health index; The self-learning fault prediction model is used to obtain a random forest model based on historical inspection data training, and use the random forest model to predict the failure probability of the equipment.

[0012] In a specific implementation scheme, the dynamic scheduling layer includes: An inspection route planning module is used to dynamically generate an optimal inspection sequence based on the equipment health index, usage frequency, and historical failure rate; The priority queue management module is used to put devices with high failure probability at the top and delay inspection of low-priority devices.

[0013] In a specific implementation scheme, the execution and feedback layer includes: An automated repair module, used for remotely restarting, upgrading firmware, and calibrating parameters of the multimedia device; Visual reporting platform for generating interactive reports.

[0014] In a second aspect, a multi-modal multimedia device automatic inspection method is provided, comprising the following steps: Use the data collection layer to obtain multimodal data and operation logs of multimedia devices; Generate a device health index using an intelligent analysis layer and predict the failure probability of the multimedia device; The dynamic scheduling layer is used to perform inspection path planning and priority queue management on the multimedia device.

[0015] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0016] In a specific embodiment, it also includes: The execution and feedback layer is used to repair the multimedia device and generate a visual report.

[0017] In a third aspect, an electronic device is provided, comprising a processor, the processor being coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor so that the electronic device implements any one of the multimodal multimedia device automatic inspection methods.

[0018] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0019] In a fourth aspect, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements any one of the multimodal-based multimedia device automatic inspection methods.

[0020] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A structural block diagram of a multi-modal multimedia device automatic inspection system provided in an embodiment of the present application; Figure 2 A flowchart of a multi-modal multimedia device automatic inspection method provided in an embodiment of the present application; Figure 3 A specific flow chart of the multi-modal multimedia device automatic inspection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.

[0023] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0024] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0025] To facilitate understanding of the multi-modal automatic inspection system and method for multimedia devices provided in the embodiment of the present application, its application scenario is first explained. The multi-modal automatic inspection system and method for multimedia devices provided in the embodiment of the present application are used to improve the comprehensiveness of problem identification. The existing multimedia device inspection system has the following defects: Low efficiency of manual inspection: The existing technology relies on manual regular inspection, which is easy to miss hidden problems (such as circuit aging, software compatibility failure). Static inspection plan: The traditional solution adopts fixed-cycle inspection (such as once a day), which cannot be dynamically adjusted according to the actual frequency of use of the equipment, resulting in waste of resources. Single-dimensional diagnosis: Only judging the fault through the equipment status indicator or a simple self-test program, it is impossible to identify complex problems (such as audio distortion, video frame loss). Lack of predictive maintenance: It is impossible to predict potential faults in advance, resulting in sudden equipment downtime. For this reason, the embodiment of the present application provides a multi-modal automatic inspection system and method for multimedia devices to improve the comprehensiveness of problem identification. The following is a detailed description of the embodiment in conjunction with specific drawings.

[0026] refer to Figures 1 to 3 , Figure 1 A structural block diagram of a multi-modal multimedia device automatic inspection system provided in an embodiment of the present application; Figure 2 A flowchart of a multi-modal multimedia device automatic inspection method provided in an embodiment of the present application; Figure 3 A specific flow chart of the multi-modal multimedia device automatic inspection method provided in an embodiment of the present application.

[0027] exist Figure 1 In the embodiment of the present application, a multi-modal multimedia device automatic inspection system is provided, comprising: Data collection layer, used to obtain multimodal data and operation logs of multimedia devices; An intelligent analysis layer, for generating a device health index and predicting a failure probability of the multimedia device; The dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia devices.

[0028] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0029] In a specific implementation scheme, it also includes: an execution and feedback layer, wherein: The execution and feedback layer is used to repair the multimedia device and generate a visual report.

[0030] In a specific implementation scheme, the data collection layer includes: Multimodal sensors, including an integrated audio acquisition module, video quality analysis unit and hardware sensors; Device log interface, used to obtain device operation logs in real time.

[0031] In a specific implementation scheme, the intelligent analysis layer includes: Multimodal data fusion engine, which is used to integrate audio, video, and sensor data to generate device health index; The self-learning fault prediction model is used to obtain a random forest model based on historical inspection data training, and use the random forest model to predict the failure probability of the equipment.

[0032] In a specific implementation scheme, the dynamic scheduling layer includes: An inspection route planning module is used to dynamically generate an optimal inspection sequence based on the equipment health index, usage frequency, and historical failure rate; The priority queue management module is used to put devices with high failure probability at the top and delay inspection of low-priority devices.

[0033] In a specific implementation scheme, the execution and feedback layer includes: An automated repair module, used for remotely restarting, upgrading firmware, and calibrating parameters of the multimedia device; Visual reporting platform for generating interactive reports.

[0034] Specifically, refer to Figure 1 and Figure 3 The multi-modal multimedia equipment automatic inspection system specifically includes: 1. Data collection layer: Multimodal sensor: integrated audio acquisition module (detection of signal-to-noise ratio and distortion), video quality analysis unit (detection of resolution and frame rate stability), and hardware sensors (temperature, current, and vibration).

[0035] Device log interface: obtain device operation logs (such as firmware version, abnormal error records) in real time.

[0036] 2. Intelligent analysis layer: Multimodal data fusion engine: Integrates audio, video, and sensor data to generate a device health index (0-100 points).

[0037] Self-learning fault prediction model: A random forest model is trained based on historical inspection data to predict the probability of failure within the next 72 hours.

[0038] 3. Dynamic scheduling layer: Inspection route planning module: dynamically generates the optimal inspection sequence based on equipment health index, usage frequency, and historical failure rate.

[0039] Priority queue management module: puts devices with high failure probability at the top and delays inspection of low-priority devices.

[0040] 4. Execution and feedback layer: Automatic repair module: supports remote restart, firmware upgrade, parameter calibration and other operations.

[0041] Visual reporting platform: Generates interactive reports including problem location, repair records, and maintenance recommendations.

[0042] Furthermore, multimodal perception and health scoring include: Audio quality analysis: Detect microphone distortion and identify sudden changes in background noise (such as electric current sound) through voiceprint comparison.

[0043] Video performance evaluation: Analyze the resolution stability, color deviation, and frame rate fluctuation of the video stream, and mark abnormal devices.

[0044] Sensor data fusion: Combine temperature, current fluctuations and vibration frequency to determine hardware aging or poor contact.

[0045] Health index calculation: Generate a comprehensive score based on weighted multi-dimensional data (e.g. audio accounts for 30%, video accounts for 40%, and sensors account for 30%).

[0046] Furthermore, the dynamic path planning module includes: Real-time priority adjustment: including: High-frequency use equipment: mandatory daily in-depth inspection of the main camera in the conference room; historically high-failure equipment: automatically shorten the inspection cycle (such as changing from weekly to daily).

[0047] Geographic optimization: Cluster distributed equipment (such as multiple conference rooms) by physical location to reduce mobile inspection time.

[0048] Furthermore, the self-learning fault prediction model includes: Feature engineering: Inputs include device model, cumulative working hours, number of recent failures, ambient humidity, etc.

[0049] Prediction output: Output failure probability and possible failure types (such as hardware damage, software conflict).

[0050] Model iteration: Update training data every quarter to improve prediction accuracy.

[0051] Furthermore, cross-device collaborative diagnosis includes: Data sharing mechanism: Devices of the same model share failure modes, allowing common defects to be quickly located (e.g. a batch of microphones is susceptible to moisture).

[0052] Remote expert support: Complex problems automatically trigger remote diagnosis requests, and experts guide on-site maintenance through AR annotations.

[0053] In a specific embodiment, the camera in a certain company's conference room frequently freezes. The implementation process is as follows: 1. Data collection: The video analysis module detected that the frame rate dropped from 30fps to 15fps and the color deviation value exceeded the limit.

[0054] The temperature sensor shows that the device temperature is continuously above 45°C.

[0055] The log shows recent firmware upgrade failure records.

[0056] 2. Health score: Video performance score: 60 (low frame rate + abnormal color), temperature score: 40, and comprehensive health index: 53 (warning threshold: 55).

[0057] 3. Dynamic Scheduling: The system raises the priority of the camera to the first place, triggering instant depth detection.

[0058] 4. Fault prediction and repair: The model predicts that the probability of "thermal failure" is 85%, and it is recommended to clean the fan and downgrade the firmware.

[0059] The automation module remotely downgrades the firmware and notifies maintenance personnel to clean the fans on site.

[0060] 5. Report Generation: The generated report shows "Poor heat dissipation leads to performance degradation", with cleaning tutorials and firmware version recommendations attached.

[0061] In the above technical solution, the beneficial effects include: 1. Multimodal fusion diagnosis: Combines audio and video performance with hardware sensor data to improve the comprehensiveness of problem identification.

[0062] 2. Dynamic scheduling mechanism: adjust the inspection plan in real time according to the equipment status to optimize resource utilization.

[0063] 3. Predictive maintenance: Early warning of potential failures to reduce the risk of sudden downtime.

[0064] 4. Cross-device collaboration: Improve efficiency in solving complex problems through data sharing and remote support.

[0065] exist Figure 2 In the embodiment of the present application, a method for automatic inspection of multimedia devices based on multi-modality is provided, comprising the following steps: Use the data collection layer to obtain multimodal data and operation logs of multimedia devices; Generate a device health index using an intelligent analysis layer and predict the failure probability of the multimedia device; The dynamic scheduling layer is used to perform inspection path planning and priority queue management on the multimedia device.

[0066] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0067] In a specific embodiment, it also includes: The execution and feedback layer is used to repair the multimedia device and generate a visual report.

[0068] An embodiment of the present application also provides an electronic device, comprising a processor, the processor being coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor so that the electronic device implements any one of the multimodal multimedia device automatic inspection methods.

[0069] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0070] An embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that the computer-readable storage medium implements any one of the multi-modal multimedia device automatic inspection methods.

[0071] In the above technical solution, a data collection layer is set up to obtain multimodal data and operation logs of multimedia devices; an intelligent analysis layer is used to generate a device health index and predict the failure probability of the multimedia device; and a dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia device; thereby improving the comprehensiveness of problem identification during the automatic inspection of multimedia devices.

[0072] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.

[0073] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.

[0074] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0075] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.

Claims

1. A multi-modal multimedia equipment automatic inspection system, characterized in that: include: Data collection layer, used to obtain multimodal data and operation logs of multimedia devices; An intelligent analysis layer, for generating a device health index and predicting a failure probability of the multimedia device; The dynamic scheduling layer is used to perform inspection path planning and priority queue management for the multimedia devices.

2. The multi-modal multimedia device automatic inspection system according to claim 1, characterized in that: Also includes: Execution and feedback layer, where The execution and feedback layer is used to repair the multimedia device and generate a visual report.

3. The multi-modal multimedia device automatic inspection system according to claim 2, characterized in that: The data collection layer includes: Multimodal sensors, including an integrated audio acquisition module, video quality analysis unit and hardware sensors; Device log interface, used to obtain device operation logs in real time.

4. The multi-modal multimedia device automatic inspection system according to claim 3 is characterized in that: The intelligent analysis layer includes: Multimodal data fusion engine, which is used to integrate audio, video, and sensor data to generate device health index; The self-learning fault prediction model is used to obtain a random forest model based on historical inspection data training, and use the random forest model to predict the failure probability of the equipment.

5. The multi-modal multimedia device automatic inspection system according to claim 4 is characterized in that: The dynamic scheduling layer includes: An inspection route planning module is used to dynamically generate an optimal inspection sequence based on the equipment health index, usage frequency, and historical failure rate; The priority queue management module is used to put devices with high failure probability at the top and delay inspection of low-priority devices.

6. The multi-modal multimedia device automatic inspection system according to claim 5, characterized in that: The execution and feedback layer includes: An automated repair module, used for remotely restarting, upgrading firmware, and calibrating parameters of the multimedia device; Visual reporting platform for generating interactive reports.

7. A multi-modal multimedia device automatic inspection method, characterized in that: The following steps are involved: Use the data collection layer to obtain multimodal data and operation logs of multimedia devices; Generate a device health index using an intelligent analysis layer and predict the failure probability of the multimedia device; The dynamic scheduling layer is used to perform inspection path planning and priority queue management on the multimedia device.

8. The multi-modal multimedia device automatic inspection method according to claim 7, characterized in that: Also includes: The execution and feedback layer is used to repair the multimedia device and generate a visual report.

9. An electronic device, characterized in that: The electronic device includes a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the multi-modal multimedia device automatic inspection method as described in any one of claims 7 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer-readable storage medium implements the multi-modal multimedia device automatic inspection method as described in any one of claims 7 to 8.

Images