Log monitoring in-vehicle multi-screen recording method and system and electronic equipment

Abstract

The invention relates to the field of video recording, in particular to a log monitoring in-vehicle multi-screen recording method and system and an electronic device.The log monitoring in-vehicle multi-screen recording method comprises the steps that the number of display devices interacting with a signaling server is obtained, virtual screens of the multiple display devices are intercepted, and mirror image areas matched with the number of the display devices are created on the virtual screens; when the signaling server receives a screen recording operation instruction triggered by a user side, automatic screen monitoring is started to obtain a monitoring log, and a multi-scene test case set based on the monitoring log is generated; all the test cases in the multi-scene test case set are executed, and execution results are displayed in the virtual screen in real time; collecting display data of a plurality of virtual screens, and outputting the display data to corresponding mirror image areas in a mirroring manner; circularly recording the projection content displayed on the mirror image area to synthesize a video; and uploading the synthesized video to a cloud end to generate a playback file. According to the scheme, when abnormal operation query is carried out on a user side, the video file obtained by screen recording is played back, and the problem that operation errors are difficult to reproduce is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of video recording, and in particular to a vehicle multi-screen recording method, system, electronic device and storage medium for log monitoring. Background Art

[0002] At present, as cars become more and more intelligent, especially new energy vehicles are becoming more and more mainstream domestic automobile products, the monitoring of the car's central control system mainly records the execution of programs through system logs, which is also a common method used by many car companies using Android smart systems.

[0003] However, in some scenarios, it is difficult to analyze and judge the screen display only through logs. For example, the logs obtained seem to be fine, but there is a problem with the screen display at the time. This situation is also quite common. In order to solve program problems, R&D personnel often need to communicate with on-site personnel and try to obtain valid logs. In order to avoid ambiguity or deviation in understanding, they record videos on their mobile phones or repeat operations to reproduce the problem and locate the cause, which is convenient for R&D personnel to track and provide feedback. However, if the problem is difficult to reproduce and capture, or the video recorded by the mobile phone is not clear enough to restore the scene, etc., it will cause certain troubles in analyzing and solving the problem.

[0004] Some car companies provide screen recording functions in the same way as mobile phones, but they also record events in the future. This method is limited by the size of storage space, the recording time is limited, and low-probability occasional problems are difficult to capture and save. Therefore, like a dash cam, effectively recording and saving the status of the car screen for historical events that have already occurred has always been an urgent need for car company R&D personnel and car owners.

[0005] The existing recording methods are mainly through the following means:

[0006] 1. Use a mobile phone to record a video: It can record the problems that must occur, but it is difficult to capture occasional problems, and the operation time is long, so the problem cannot be captured immediately;

[0007] 2. It can capture the problem again after it occurs, but occasionally the problem cannot be located;

[0008] 3. Export system log: This is a common way to record problems, but it only contains text information and no video content, making it difficult to locate the problem. Summary of the invention

[0009] In order to remedy the above-mentioned defects, the purpose of the present invention is to provide a multi-screen recording method, system and electronic device for log monitoring of a vehicle, which can automatically record the log of an application and capture the problems existing in the log at the first time, thereby solving the defect that the recording of the display content only has text information but no video content, making it difficult to accurately locate and find operational accidents or abnormalities.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] In a first aspect, the present application provides a vehicle multi-screen recording method for log monitoring, the method comprising:

[0012] Acquire the number of display devices interacting with the signaling server, capture virtual screens of multiple display devices, and create a mirror area matching the number of display devices on the virtual screen;

[0013] When the signaling server receives a screen recording operation instruction triggered by the user end through the screen recording service application, it starts automatic screen monitoring for the screen recording operation instruction triggered by the user end, obtains the monitoring log, and generates a multi-scenario test case set based on the monitoring log; executes all test cases in the multi-scenario test case set, and displays the execution results in real time on the virtual screen;

[0014] Collect display data of multiple virtual screens and output them to corresponding mirrored areas; loop record the projection content displayed on the mirrored areas and synthesize the video;

[0015] After recording, the synthesized video is uploaded to the cloud and a playback file is generated, which the user can use to obtain the playback file from the cloud and locate problems in the monitoring log.

[0016] Optionally, obtaining monitoring logs and generating a multi-scenario test case set includes: monitoring the logs generated by the interaction of the display device that interacts with the signaling server, and generating a single test scenario log set based on the relationship between the display device and the signaling server; extracting data fields in the single test scenario message log set to generate a test case set for a single test scenario; marking the test scenario type corresponding to the test case set for the single test scenario; and combining all the generated single test scenario test case sets to generate a multi-scenario test case set.

[0017] Optionally, the screen recording operation instruction is an instruction output by the user terminal by setting the video image size, frame rate, and encoder setting parameters; wherein the screen recording operation instruction includes a partial synthesis instruction carrying the synthesis time or a full video synthesis instruction.

[0018] Optionally, the loop recording of the projection content displayed on the mirror area, the synthesized video includes:

[0019] The media projection API interface pre-introduced in the screen recording service application is called to capture the projection screen displayed on the mirror area, encode and loop-record the projection screen, and obtain the video information; the video information is converted into an HDMI signal and transmitted to the signaling server for local cache, and the video information in the cache is synthesized to form a video file.

[0020] Optionally, synthesizing the video information in the cache to form a video file includes:

[0021] After receiving the screen recording operation instruction, the video information in the HDMI signal is written to obtain audio and video stream data; wherein the audio and video stream data includes image sequence, audio and subtitle data;

[0022] The audio and subtitle data of the image sequence are compressed after being synchronized with the timestamp; the compressed image sequence, audio and subtitle data are synthesized into a video to form a video file; when multiple audio and subtitle data exist at the same time, the audio and subtitle are added to different channels.

[0023] Optionally, the recording information written into the HDMI signal to obtain the audio and video stream data includes: receiving the HDMI signal, extracting the image size, frame rate information, RGB or YUV channel image data from the TMDS clock signal and the recording information of the HDMI signal, and synthesizing an RGB image sequence;

[0024] According to the TDCS / CTC clock and image frame rate, the actual time when the image is generated is calculated and annotated to generate an image sequence; the audio and subtitle data corresponding to the image sequence are obtained.

[0025] Optionally, the method of obtaining audio and subtitle data corresponding to the image sequence includes: receiving voice data in the HDMI signal, converting the voice into text as video subtitles; and setting the corresponding playback time of the subtitles according to the system time when the voice starts and the voice parameters, identifying the speaker according to the recording device ID, and adding it to the subtitles.

[0026] Optionally, generating the playback file includes: defining the playback file as a file format consisting of a message header and a message body, and synthesizing the video based on the file format of the playback file to complete the generation of the playback file;

[0027] The message header is used to record basic information of the synthesized video, including the start time, duration, and number of entries;

[0028] The message body is used to record the execution result data content of the test cases of audio, video, video subtitles, and monitoring logs.

[0029] In a second aspect, the present application also provides a vehicle multi-screen recording system for log monitoring, the system comprising:

[0030] A processing module, used to obtain the number of display devices interacting with the signaling server, and capture virtual screens of multiple display devices, and create a mirror area matching the number of display devices on the virtual screen;

[0031] The test case execution module is used to start automatic screen monitoring for the screen recording operation instruction triggered by the user end, obtain the monitoring log, and generate a multi-scenario test case set based on the monitoring log when the signaling server receives the screen recording operation instruction triggered by the user end through the screen recording service application; execute all test cases in the multi-scenario test case set, and display the execution results in real time on the virtual screen;

[0032] The video recording and synthesis module is used to collect display data of multiple virtual screens and output them in a mirrored manner to the corresponding mirrored area; the projection content displayed on the mirrored area is recorded in a loop to synthesize the video;

[0033] The playback module is used to upload the synthesized video to the cloud after recording is completed and generate a playback file for the user to obtain the playback file from the cloud and locate the problem in the monitoring log.

[0034] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the method steps as described in any one of the first aspects.

[0035] The beneficial effects of the present invention are embodied in:

[0036] The present invention provides a log-monitored vehicle multi-screen recording method, system and electronic device, which include obtaining the number of display devices interacting with a signaling server, capturing the virtual screens of multiple display devices, and creating a mirror area on the virtual screen that matches the number of display devices; when the signaling server receives a screen recording operation instruction triggered by a user end, it starts automatic screen monitoring to obtain a monitoring log, and generates a multi-scenario test case set based on the monitoring log; executes all test cases in the multi-scenario test case set, and displays the execution results in real time on the virtual screen; shortens the test steps and improves the test efficiency. At the same time, the display data of multiple virtual screens are collected and mirrored to the corresponding mirror area; the projection content displayed on the mirror area is recorded in a loop to synthesize the video; the synthesized video is uploaded to the cloud to generate a playback file. The above scheme can play back the video file obtained by screen recording when the user end performs an abnormal operation query, which makes up for the problem that the operation abnormality is difficult to reproduce.

[0037] The present invention provides a log-monitored vehicle multi-screen recording method, system and electronic device, which can clearly record the faults of the central control system by multi-screen recording, which is not only convenient for problem solving during the R&D process of the vehicle company, but also for communication and problem solving with after-sales personnel; it can also be preserved as evidence in certain scenarios, which is convenient for dealing with various emergencies. This can help shorten the workflow of vehicle company testers and improve testing efficiency; it helps improve the efficiency of handling after-sales problems and improve user satisfaction; especially when disputes are involved, it can be preserved as evidence for future reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0039] Figure 1 A flow chart of a vehicle multi-screen recording method for log monitoring provided by the present invention;

[0040] Figure 2 A schematic diagram of the principle of a single-screen recording technology provided by the present invention;

[0041] Figure 3 A schematic diagram of the principle of a multi-screen recording technology provided by the present invention;

[0042] Figure 4 A structural schematic diagram of a vehicle multi-screen recording system for log monitoring provided by the present invention;

[0043] Figure 5 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0045] The embodiments of the present application provide a method, system and electronic device for multi-screen recording of a vehicle machine for log monitoring, which can be applied to a terminal, a signaling server, or a system including a terminal and a signaling server, and is implemented through the interaction between the terminal and the signaling server. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc.

[0046] The implementation scenario of the embodiment of the present application is mainly used for the host system with a screen in the automobile system, especially for automobiles with more screens, such as the car system including instrument screen projection, HUD screen projection, and DLP matrix headlight screen projection, to facilitate recording the display status of each screen. And other scenarios, including;

[0047] Educational applications: Teachers can record teaching videos, and students can record the learning process for later reference.

[0048] Video Recording: Allows players to record exciting game moments and share them with friends or on social media.

[0049] Product Demonstration: Software developers can record application operation tutorials to allow users to understand the functions more intuitively.

[0050] Remote collaboration: Team members can record their screens to clearly demonstrate problems or share solutions.

[0051] The present invention provides a multi-screen recording method for a vehicle computer with log monitoring. The technical solution adopted can automatically record the log of the application program, and capture the problems existing in the log in the first place, mainly solving the defect that it is difficult to accurately locate and find the operation accident or abnormality due to the recording of the display content having only text information but no video content.

[0052] The embodiments of the present invention are described below in conjunction with the accompanying drawings.

[0053] Example 1: Please refer to Figure 1 Embodiment 1 of the present invention provides a vehicle multi-screen recording method for log monitoring, and the specific steps of the method include:

[0054] S101 obtains the number of display devices interacting with the signaling server, captures virtual screens of multiple display devices, and creates a mirror area matching the number of display devices on the virtual screen;

[0055] S102: when the signaling server receives a screen recording operation instruction triggered by the user terminal through the screen recording service application, it starts automatic screen monitoring for the screen recording operation instruction triggered by the user terminal, obtains the monitoring log, and generates a multi-scenario test case set based on the monitoring log; executes all test cases in the multi-scenario test case set, and displays the execution results in real time on the virtual screen;

[0056] S103 collects display data of multiple virtual screens and outputs them to corresponding mirrored areas; loops and records the projection content displayed on the mirrored areas to synthesize the video;

[0057] After the recording is completed in S104, the synthesized video is uploaded to the cloud and a playback file is generated, so that the user end can obtain the playback file from the cloud and locate the problem in the monitoring log.

[0058] In the above embodiment, step S101 of obtaining monitoring logs and generating a multi-scenario test case set includes: monitoring the logs generated by the interaction of the display device interacting with the signaling server, and generating a single test scenario log set according to the relationship between the display device and the signaling server; extracting data fields in the single test scenario message log set to generate a test case set for a single test scenario; marking the test scenario type corresponding to the test case set for the single test scenario; and combining all the generated single test scenario test case sets to generate a multi-scenario test case set.

[0059] In the above embodiment, the screen recording operation instruction of step S102 is an instruction output by the user end by setting the video image size, frame rate, and encoder setting parameters; wherein the screen recording operation instruction includes a partial synthesis instruction carrying the synthesis time or a full video synthesis instruction.

[0060] In the above embodiment, step S103 of synthesizing video describes in detail the detailed steps of synthesizing video of the recording screen.

[0061] In the prior art, reference Figure 2 As shown in the figure, in the normal monitoring of a single main screen scenario, the Android system media projection API image projection method is used to capture the Surface, the image is output to a virtual screen Virtual Display for display, and then converted into a Surface that can be recorded by the Media Recorder for recording. At the same time, the Audio Recorder can be used to capture sound data and synthesize the final mp4 video file.

[0062] Screen record is a shell command of the Android system. How to make screen record run on the Linux system is the first problem that must be solved. This requires modifying, compiling and running the Android source code under Ubuntu. The workload is large. First, you need to modify the environment variables and compile the tool chain so that the source code can pass normally. In order to be able to use the screen record program in the Linux environment, so as to achieve the effect of screen recording.

[0063] However, this video file is relatively short and is a video clip that is regularly stored on the device. The circular storage method is used to ensure that the total amount of video does not exceed the threshold and expired files are deleted at any time.

[0064] Therefore, in order to ensure that the screen recording content within several hours can be recorded, when exporting the recorded content, the video clips can be combined into a long video of the required time to provide it to users. Many car computers are designed with more than one screen, which may correspond to multiple user scenarios, or use the screen projection function to map to other screens for display. In this case, the above design cannot meet the requirements well, and the images need to be synthesized into one screen for unified output.

[0065] Therefore, the above problem can be solved by looping and recording the projection content displayed on the mirror area to synthesize the video in step S103 in the embodiment of the present invention. The execution process specifically includes:

[0066] The media projection API interface pre-introduced in the screen recording service application is called to capture the projection screen displayed on the mirror area, encode and loop-record the projection screen, and obtain the video information; the video information is converted into an HDMI signal and transmitted to the signaling server for local cache, and the video information in the cache is synthesized to form a video file.

[0067] In the above embodiment, H.264 protocol can be used for encoding and decoding. The use of H.264 protocol has relatively high requirements for the encoding and decoding algorithm and causes greater damage to the video. If you want to pursue smoothness, you must sacrifice the clarity of the image quality and increase the compression ratio, which will increase the amount of transmitted data and cause jamming. Therefore, developers must calculate clearly to keep the clarity within a reasonable range and ensure smoothness.

[0068] In the above embodiment, synthesizing the video information in the cache to form a video file includes:

[0069] After receiving the screen recording operation instruction, the video information in the HDMI signal is written to obtain audio and video stream data; wherein the audio and video stream data includes image sequence, audio and subtitle data;

[0070] The audio and subtitle data of the image sequence are compressed after being synchronized with the timestamp; the compressed image sequence, audio and subtitle data are synthesized into a video to form a video file; when multiple audio and subtitle data exist at the same time, the audio and subtitle are added to different channels.

[0071] In the above embodiment, the recording information written into the HDMI signal to obtain the audio and video stream data includes: receiving the HDMI signal, extracting the image size, frame rate information, RGB or YUV channel image data from the TMDS clock signal and the recording information of the HDMI signal, and synthesizing an RGB image sequence;

[0072] According to the TDCS / CTC clock and image frame rate, the actual time when the image is generated is calculated and annotated to generate an image sequence; the audio and subtitle data corresponding to the image sequence are obtained.

[0073] In the above embodiment, the acquisition of audio and subtitle data corresponding to the image sequence includes: receiving voice data in the HDMI signal, converting the voice into text as video subtitles; and setting the corresponding playback time of the subtitles according to the system time when the voice starts and the voice parameters, identifying the speaker according to the recording device ID, and adding it to the subtitles.

[0074] It is understandable that the above embodiment is different from the monitoring scenario of a single screen, and multi-screen monitoring is realized through the above content; the principle of the multi-screen monitoring scenario is as follows Figure 3 shown.

[0075] First, we need to obtain the number of display devices, capture the corresponding Surfaces and combine them into the same window, display them on the virtual display, convert them into a surface that can be recorded by the media recorder, and finally synthesize the video. The other principles are the same as those of single-screen monitoring recording.

[0076] The video generated by the above method can effectively capture the problem points along with the log information, which facilitates the troubleshooting and location of the problem. When the user performs an abnormal operation query, the video file obtained by the screen recording can be played back, which makes up for the problem that the operation abnormality is difficult to reproduce.

[0077] Furthermore, the generation of the playback file in step S104 includes: defining the playback file as a file format consisting of a message header and a message body, and synthesizing the video based on the file format of the playback file to complete the generation of the playback file;

[0078] The message header is used to record basic information of the synthesized video, including the start time, duration, and number of entries;

[0079] The message body is used to record the execution result data content of the test cases of audio, video, video subtitles, and monitoring logs.

[0080] When the playback file is transferred, the business signaling server will temporarily store the file on the local disk of the transit signaling server, using the globally unique conference number as the key and .rec as the suffix, and upload it to the cloud through the cloud service API. After the storage is successful, the local file will be deleted at the same time, otherwise it will be retained and the status will be returned to the upload process;

[0081] When the playback file is transferred to the cloud signaling server, the backend web signaling server is notified that the file has been successfully uploaded. The backend service records the file URL corresponding to the conference number into the database and returns the status to the upload process.

[0082] The player parses the audio, video and subtitle data, and based on the audio timeline, the video and subtitle data are synchronized with the audio timeline according to relative timestamps, and the playback file is drawn and displayed on the player.

[0083] In addition, when analyzing monitoring logs based on playback files, faults in the monitoring logs of the playback files can be analyzed based on single images or image sequences and voices, and the analysis results can be given for operators or managers to review.

[0084] Specifically, it can receive image sequences, voice and subtitle information from multiple terminals and perform recognition according to predefined judgment principles. By scanning the image sequence, identifying the information in the video, extracting the equipment status information, operation information and time; identifying the type and number of signal equipment faults, and recording the time when the fault started; identifying the dispatch command drafting, issuing, signing operations and dispatch command types, and recording the corresponding time; for abnormal operations of equipment failures, the equipment failure time is used as the starting point. At the same time, the operation action sets from different terminals can also be associated to analyze the coordination of different services.

[0085] The operation action set is compared with the normal operation process to form an analysis result. The analysis result information includes the time when the operation occurred, the corresponding device, specific operation, operator, analysis conclusion and terminal ID corresponding to the video, and the analysis result is stored; multiple audio and video files corresponding to each terminal belonging to the same vehicle and navigation location are grouped together and stored in chronological order to obtain the analysis result.

[0086] For the method steps disclosed in the above embodiments, the method steps are described as a series of action combinations for the purpose of simple description, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0087] Based on the same inventive concept, an embodiment of the present application also provides a vehicle-mounted multi-screen recording system for log monitoring for implementing the above-mentioned vehicle-mounted multi-screen recording method for log monitoring. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more vehicle-mounted multi-screen recording system embodiments for log monitoring provided below can be found in the above limitations on the above-mentioned vehicle-mounted multi-screen recording method for log monitoring, and will not be repeated here. It can be understood that the above description of its method is also applicable to the description of the system.

[0088] In one embodiment, the present invention also provides a vehicle multi-screen recording system for log monitoring, such as Figure 4 As shown, it includes: a processing module 11, a test case execution module 12, a video recording and synthesis module 13 and a playback module 14, wherein:

[0089] The processing module 11 is used to obtain the number of display devices interacting with the signaling server, and capture the virtual screens of multiple display devices, and create a mirror area matching the number of display devices on the virtual screen;

[0090] The test case execution module 12 is used to start automatic screen monitoring for the screen recording operation instruction triggered by the user end, obtain the monitoring log, and generate a multi-scenario test case set based on the monitoring log when the signaling server receives the screen recording operation instruction triggered by the user end through the screen recording service application; execute all test cases in the multi-scenario test case set, and display the execution results in real time on the virtual screen;

[0091] The video recording and synthesis module 13 is used to collect display data of multiple virtual screens and output them in a mirror image to the corresponding mirror image area; loop recording the projection content displayed on the mirror image area to synthesize the video;

[0092] The playback module 14 is used to upload the synthesized video to the cloud after recording is completed, generate a playback file, and allow the user end to obtain the playback file from the cloud and locate the problem in the monitoring log.

[0093] At the same time, the present application also proposes a computer-readable storage medium and an electronic device. In one embodiment, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the methods in step S101 to step S104 are implemented.

[0094] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the method described in any one of S101 to S104 is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0095] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0096] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A vehicle multi-screen recording method for log monitoring, characterized in that: The method comprises: Acquire the number of display devices interacting with the signaling server, capture virtual screens of multiple display devices, and create a mirror area matching the number of display devices on the virtual screen; When the signaling server receives a screen recording operation instruction triggered by the user end through the screen recording service application, it starts automatic screen monitoring for the screen recording operation instruction triggered by the user end, obtains the monitoring log, and generates a multi-scenario test case set based on the monitoring log; executes all test cases in the multi-scenario test case set, and displays the execution results in real time on the virtual screen; Collect display data of multiple virtual screens and output them to corresponding mirrored areas; loop record the projection content displayed on the mirrored areas and synthesize the video; After recording, the synthesized video is uploaded to the cloud and a playback file is generated, which the user can use to obtain the playback file from the cloud and locate problems in the monitoring log.

2. The method according to claim 1, characterized in that The obtaining of monitoring logs and generating a multi-scenario test case set includes: monitoring the logs generated by the interaction of the display device interacting with the signaling server, and generating a single test scenario log set according to the relationship between the display device and the signaling server; extracting data fields in the single test scenario message log set to generate a test case set for a single test scenario; marking the test scenario type corresponding to the test case set for the single test scenario; and combining all the generated single test scenario test case sets to generate a multi-scenario test case set.

3. The method according to claim 1, characterized in that The screen recording operation instruction is an instruction output by the user end by setting the video image size, frame rate, and encoder setting parameters; wherein, the screen recording operation instruction includes a partial synthesis instruction carrying the synthesis time or a full video synthesis instruction.

4. The method according to claim 1, characterized in that The loop recording of the projection content displayed on the mirror area, the composite video includes: The media projection API interface pre-introduced in the screen recording service application is called to capture the projection screen displayed on the mirror area, encode and loop-record the projection screen, and obtain the video information; the video information is converted into an HDMI signal and transmitted to the signaling server for local cache, and the video information in the cache is synthesized to form a video file.

5. The method according to claim 4, characterized in that The step of synthesizing the video information in the cache to form a video file includes: After receiving the screen recording operation instruction, the video information in the HDMI signal is written to obtain audio and video stream data; wherein the audio and video stream data includes image sequence, audio and subtitle data; The audio and subtitle data of the image sequence are compressed after being synchronized with the timestamp; the compressed image sequence, audio and subtitle data are synthesized into a video to form a video file; when multiple audio and subtitle data exist at the same time, the audio and subtitle are added to different channels.

6. The method according to claim 5, characterized in that The recording information written into the HDMI signal to obtain the audio and video stream data comprises: receiving the HDMI signal, extracting the image size, frame rate information, RGB or YUV channel image data from the TMDS clock signal and the recording information of the HDMI signal, and synthesizing an RGB image sequence; According to the TDCS / CTC clock and image frame rate, the actual time when the image is generated is calculated and annotated to generate an image sequence; the audio and subtitle data corresponding to the image sequence are obtained.

7. The method according to claim 6, characterized in that The method of obtaining audio and subtitle data corresponding to the image sequence includes: receiving voice data in the HDMI signal, converting the voice into text as video subtitles; and setting the corresponding playback time of the subtitles according to the system time when the voice starts and the voice parameters, identifying the speaker according to the recording device ID, and adding the speaker to the subtitles.

8. The method according to claim 1, characterized in that Generating the playback file includes: defining the playback file as a file format consisting of a message header and a message body, synthesizing the video based on the file format of the playback file, and completing the generation of the playback file; The message header is used to record basic information of the synthesized video, including the start time, duration, and number of entries; The message body is used to record the execution result data content of the test cases of audio, video, video subtitles, and monitoring logs.

9. A multi-screen recording system for log monitoring, characterized in that: The system comprises: A processing module, used to obtain the number of display devices interacting with the signaling server, and capture virtual screens of multiple display devices, and create a mirror area matching the number of display devices on the virtual screen; The test case execution module is used to start automatic screen monitoring for the screen recording operation instruction triggered by the user end, obtain the monitoring log, and generate a multi-scenario test case set based on the monitoring log when the signaling server receives the screen recording operation instruction triggered by the user end through the screen recording service application; execute all test cases in the multi-scenario test case set, and display the execution results in real time on the virtual screen; The video recording and synthesis module is used to collect display data of multiple virtual screens and output them in a mirrored manner to the corresponding mirrored area; the projection content displayed on the mirrored area is recorded in a loop to synthesize the video; The playback module is used to upload the synthesized video to the cloud after recording is completed and generate a playback file for the user to obtain the playback file from the cloud and locate the problem in the monitoring log.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method steps as claimed in any one of claims 1 to 8.

Images