Screen jitter non-intrusive detection method and system and medium

Abstract

The invention relates to a screen jitter non-intrusive detection method and system and a medium, and relates to the technical field of automotive electronics, and the method comprises the steps: collecting image data of a vehicle-mounted screen and collection state data of a collector; correcting the image data based on the acquisition state data so as to eliminate image distortion caused by jitter of the acquisition device; and calculating an abnormal jitter quantized value of the corrected image data based on a preset abnormal jitter algorithm, comparing the abnormal jitter quantized value with a preset jitter threshold value, and judging whether the screen has abnormal jitter or not. The non-intrusive screen abnormal jitter monitoring method can be suitable for non-intrusive screen abnormal jitter monitoring under different vehicle types and various environment conditions.

Description

Technical Field

[0001] The present invention relates to the field of automotive electronic technology, and in particular to a non-intrusive detection method, system and medium for detecting screen shake. Background Art

[0002] In the current Internet car industry, with the development and popularization of smart cockpit technology, more and more vehicle control functions are displayed and operated through the on-board screen. The stability and display effect of the on-board screen are particularly important for driving safety and user experience. However, the existing on-board screen abnormality detection mostly relies on manual observation and regular inspection. This method is not only time-consuming and labor-intensive, but also difficult to find problems in real time, resulting in the inability to trace the abnormal situation, and may affect driving safety before it is handled in time.

[0003] Although some solutions on the market currently try to detect screen anomalies by using sensors or specific hardware devices, these solutions often rely on specific hardware configurations, which not only increases costs but also limits their universality between different models or manufacturers. In addition, these systems often require a certain degree of modification to the vehicle, and this invasive installation method may interfere with or damage the vehicle's original electronic system. Summary of the invention

[0004] In view of the shortcomings of the prior art described above, an object of the present invention is to provide a non-invasive detection method, system and medium for screen jitter, which can be applicable to non-invasive abnormal screen jitter monitoring in different vehicle models and under various environmental conditions.

[0005] To achieve the above purpose, the present invention adopts the following technical solution.

[0006] In a first aspect, the present invention provides a non-intrusive method for detecting screen jitter, which adopts the following technical solution: collecting image data of a vehicle-mounted screen and collection status data of a collector; The image data is corrected based on the acquisition status data to eliminate image distortion caused by collector jitter; based on a preset abnormal jitter algorithm, an abnormal jitter quantization value of the corrected image data is calculated and compared with a preset jitter threshold to determine whether abnormal screen jitter occurs on the screen.

[0007] Furthermore, in the above-mentioned non-intrusive screen shake detection method, the correcting the image data based on the acquisition status data includes: For two consecutive frames of image data I t-1 Image frame and I t The image frames are aligned; Based on the result of the alignment process, the I t-1The image frame is corrected to obtain the current I t Image frame.

[0008] Furthermore, in the above-mentioned non-intrusive detection method for screen jitter, the image data is processed by the following method: t-1 Image frame and I t The image frames are aligned, including: The transformation matrix T is generated based on the preset objective function calculation, and the transformation matrix T is used to transform the I t-1 Image frame conversion to I t Image frames, the objective function is used to characterize the alignment accuracy and / or transformation smoothness of two consecutive image frames; The change matrix T is obtained by minimizing the objective function.

[0009] Furthermore, in the above-mentioned non-intrusive screen jitter detection method, the abnormal jitter quantization value of the corrected image data is calculated based on a preset abnormal jitter algorithm, including: Based on the preset grayscale projection method, the grayscale projection curves of the continuous frame images are compared to obtain the first jitter intensity S GP ; Based on the preset feature point matching algorithm, the feature points in the continuous frame images are matched to obtain the second jitter strength S FP ; Based on the preset frame difference method, the pixel differences in the consecutive frame images are compared to obtain the third jitter intensity S FD ; Based on a preset weighting algorithm, the first jitter strength S GP , the second jitter intensity S FP And the third jitter intensity S FD Perform weighted fusion to obtain the abnormal jitter quantization value.

[0010] Furthermore, in the above-mentioned non-intrusive detection method for screen jitter, the grayscale projection curves of consecutive frame images are compared based on the preset grayscale projection method to obtain the first jitter intensity S GP ,include: Converting the image data into a grayscale image, and calculating a grayscale projection of the grayscale image in a horizontal direction and / or a vertical direction; The difference of the grayscale projection curves between the consecutive frame images is calculated and the first jitter intensity S is obtained by statistics. GP .

[0011] Furthermore, in the above-mentioned non-intrusive detection method for screen jitter, the feature points in the continuous frame images are matched based on the preset feature point matching algorithm to obtain the second jitter intensity S FP ,include: Detecting a set of feature points in the image data based on a preset feature extraction algorithm, and generating a feature descriptor for each feature point; Match feature points between consecutive frame images, and for each pair of matched feature points, calculate the displacement vector of the feature point between the previous frame and the current frame; The displacement vectors of all matching feature points are counted to obtain the second jitter strength S FP .

[0012] Furthermore, in the above-mentioned non-intrusive detection method for screen jitter, the pixel differences in the continuous frame images are compared based on the preset frame difference method to obtain the third jitter intensity S FD ,include: Calculate the pixel differences in consecutive frame images to obtain frame difference images; Performing threshold processing on the frame difference image to remove noise differences and obtain a valid frame difference image; The effective frame difference image is counted to obtain the third jitter intensity S FD .

[0013] Furthermore, in the above-mentioned non-intrusive detection method of screen shake, the method further includes: When abnormal jitter is detected, the notification mechanism is triggered and jitter-related information is reported through a preset channel, wherein the jitter-related information includes video records during the abnormality, the precise time point when the abnormality occurred, and / or the software and hardware log information of the vehicle screen.

[0014] In a second aspect, the present invention provides a non-intrusive screen shake detection system, which adopts the following technical solution: a data acquisition device, including a collector and a motion sensor, wherein the collector is used to collect image data of a vehicle-mounted screen, and the motion sensor is used to obtain acquisition status data of the collector; The data processing device includes a correction module and a detection module. The correction module corrects the image data based on the acquisition status data to eliminate image distortion caused by the jitter of the collector. The detection module is used to calculate the abnormal jitter quantization value of the corrected image data based on a preset abnormal jitter algorithm, and compare it with a preset jitter threshold to determine whether the screen has abnormal screen jitter.

[0015] Furthermore, the above-mentioned non-intrusive screen shake detection system further includes: The data reporting device is used to trigger the notification mechanism when abnormal jitter is detected, and report jitter-related information through a preset channel, wherein the jitter-related information includes video records during the abnormality, the precise time point when the abnormality occurred, and / or the software and hardware log information of the vehicle screen.

[0016] In a third aspect, the present invention provides a readable storage medium, which adopts the following technical solution: A readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, a non-intrusive method for detecting screen jitter as described in any one of the first aspects above is implemented.

[0017] In summary, compared with the prior art, the present invention includes at least one of the following beneficial technical effects: The present invention designs a non-intrusive screen anomaly detection technology that is both economical and efficient and can be widely applied to a variety of vehicle models and environments to achieve more efficient and accurate screen shake anomaly monitoring. The technology should be able to automatically detect and report anomalies without manual intervention, thereby ensuring the continued stable operation of the vehicle screen and the safety of the driver without adding additional hardware costs and complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flowchart of a specific embodiment of a non-intrusive detection method for screen jitter of the present invention.

[0020] Figure 2 It is a flowchart of another specific embodiment of a non-intrusive screen shake detection method of the present invention.

[0021] Figure 3 It is a flowchart of another specific embodiment of a non-intrusive screen shake detection method of the present invention.

[0022] Figure 4 It is a structural schematic diagram of a specific embodiment of a non-intrusive screen shake detection system of the present invention.

[0023] Figure 5 It is a structural schematic diagram of another specific embodiment of a non-intrusive screen shake detection system of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application. In addition, it should be understood that the specific implementation methods described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0025] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments of the present application. In addition, in the following embodiments, the description of each embodiment has its own emphasis, and for parts not described in detail in one embodiment, reference can be made to the relevant description of other embodiments.

[0026] The method steps described in the embodiments of the present invention may be executed in the order described in the specific implementation manner, or the execution order of each step may be adjusted according to actual needs and on the premise that the technical problem can be solved, which will not be listed here one by one.

[0027] The car screen is a display device installed on the center console of the car, usually used to display vehicle information, navigation maps, entertainment system interface, reversing images, etc. It is an important part of the car's smart cockpit, providing information and entertainment services to drivers and passengers through touch, button or voice interaction, and is also connected to the vehicle system to control vehicle functions.

[0028] Abnormal screen jitter refers to abnormal screen shaking or jittering of the vehicle display during normal use. This jitter may be caused by vibrations generated during vehicle driving, screen hardware failure, or software errors, resulting in unstable screen display content, affecting the driver's visual experience and operating accuracy, and may endanger driving safety.

[0029] Based on the above problems, refer to Figure 1 An embodiment of the present invention provides a non-intrusive method for detecting screen jitter, comprising the following steps.

[0030] S1, collect the image data of the vehicle screen and the acquisition status data of the collector; specifically, use a camera or an industrial camera to collect the image data of the vehicle screen, and at the same time use a motion sensor or a gyroscope installed on the collector (such as a camera) to collect the acquisition status data of the device, such as position, angle, acceleration, etc. These acquisition status data can record the movement of the collector when collecting images, so as to correct the image data later and eliminate the image data distortion caused by the shaking of the external acquisition device.

[0031] S2, correcting the image data based on the acquisition status data to eliminate image distortion caused by collector jitter; specifically, correcting the image data in real time based on the acquisition status data collected from the motion sensor or gyroscope. Specific correction methods may include image registration technology, which compensates for image offset, rotation or scaling caused by collector jitter by calculating the transformation matrix between consecutive frames to keep the image stable. Another possible way is to use an image stabilization algorithm, such as using motion estimation and compensation technology to correct jitter directly in the image domain. These correction methods can be used independently or in combination to ensure that all types of image distortion caused by collector jitter are eliminated.

[0032] S3, based on the preset abnormal jitter algorithm, calculate the abnormal jitter quantization value of the corrected image data, and compare it with the preset jitter threshold to determine whether the screen has abnormal screen jitter; specifically, analyze the corrected image data through a preset abnormal jitter algorithm (such as grayscale projection method, feature point matching method, frame difference method, etc.) to calculate an abnormal jitter quantization value. Compare the abnormal jitter quantization value with the preset jitter threshold. If the abnormal jitter quantization value exceeds the jitter threshold, it is determined that the screen has abnormal jitter. This process also includes weighted averaging the calculation results of multiple algorithms to improve the accuracy and reliability of detection. The final judgment can trigger further automated operations, such as alarming or generating reports. These implementations can be used independently or in combination to ensure comprehensive and accurate detection of abnormal screen jitter.

[0033] By collecting and correcting the on-board screen image data, the image distortion caused by the collector jitter is effectively eliminated, ensuring the accuracy of the detection results; at the same time, the preset abnormal jitter algorithm is used to analyze and judge the corrected image, realizing accurate detection and real-time monitoring of screen jitter, avoiding the dependence on complex hardware and the need for vehicle modification, thus providing an economical, efficient and universal screen jitter detection solution, which helps to improve the reliability of vehicle systems and driving safety.

[0034] Further, as an implementation mode of the present invention, step S2, correcting the image data based on the acquisition status data, includes: For two consecutive frames of image data I t-1 Image frame and I t The image frames are aligned; Based on the result of the alignment process, the I t-1 The image frame is corrected to obtain the current I t Image frame.

[0035] Specifically, two consecutive frames of images I are registered by image registration technology.t-1 Image frame and I t The image frames are aligned. Specific methods may include using feature point matching technology to match and align key feature points in two frames of images; or using a motion estimation algorithm based on optical flow to achieve alignment by calculating the movement vector of each pixel point in two consecutive frames of images; or using a global transformation model, such as affine transformation, perspective transformation, etc., to calculate the best transformation matrix and align the previous frame I t-1 Align to current frame I t These methods can be used individually or in combination to ensure accurate alignment of the two image frames.

[0036] Specifically, based on the result of the alignment processing, the I t-1 Image frame correction can be t-1 The image frame is geometrically transformed by the calculated transformation matrix so that it is consistent with the current frame I in position, angle or scale. t Alignment; the transformed image can also be pixel resampled by image interpolation technology to eliminate the distortion in the alignment process and ensure the clarity and accuracy of the image. These methods can be used alone or in combination to obtain the corrected current image frame I t .

[0037] Further, as an embodiment of the present invention, two consecutive frames of image data I t-1 Image frame and I t The image frame is aligned, including: generating a transformation matrix T based on a preset objective function, wherein the transformation matrix T is used to transform the I t-1 Image frame conversion to I t The objective function is used to characterize the alignment accuracy and / or transformation smoothness of two consecutive image frames; the change matrix T is obtained by minimizing the objective function.

[0038] Specifically, in the process of image stabilization, the embodiment of the present invention uses the inter-frame image registration technology to align two consecutive frames of images. t , the image frame at the previous moment is I t-1 , in order to achieve image alignment, it is necessary to calculate an optimal transformation matrix T. This process can be completed by minimizing the following objective function: T=arg min T ||I t -T(I t-1 )|| 2 +λ||D(T)|| 2 Among them, ||I t -T(It-1 )|| 2 Indicates the current frame I t Compared with the previous frame T(I t-1 ), which is used to characterize the alignment accuracy of two consecutive frames of images; D(T) represents the smoothness of the transformation matrix T; λ is the regularization coefficient, which is used to balance the relationship between alignment accuracy and transformation smoothness.

[0039] Further, as an embodiment of the present invention, refer to Figure 2 , step S3, based on a preset abnormal jitter algorithm, calculating the abnormal jitter quantization value of the corrected image data, including: S31, based on a preset grayscale projection method, compare the grayscale projection curves of consecutive frame images to obtain a first jitter intensity S GP ; S32, based on a preset feature point matching algorithm, matching the feature points in the continuous frame images to obtain a second jitter strength S FP ; S33, based on a preset frame difference method, comparing pixel differences in consecutive frame images to obtain a third jitter intensity S FP ; S34, based on a preset weighting algorithm, the first jitter strength S GP , the second jitter intensity S FP And the third jitter intensity S FD Perform weighted fusion to obtain the abnormal jitter quantization value.

[0040] Specifically, the grayscale projection method calculates the grayscale value accumulation of the image in the horizontal or vertical direction to analyze the overall structural changes of the image. When detecting jitter, the overall displacement of the image can be effectively estimated by comparing the grayscale projection curve differences between consecutive frames. The grayscale projection method is particularly sensitive to image changes caused by translation or slight rotation.

[0041] Feature point matching focuses on identifying and tracking key feature points in an image. By matching these key points in consecutive image frames, the relative motion between images can be accurately estimated. This method is suitable for detecting complex image changes caused by camera shake, including rotation and scaling.

[0042] The frame difference method detects the motion of an image by analyzing the differences between consecutive frames. This method is very sensitive to fast movements or sudden changes in a short period of time, making it an effective tool for capturing sudden jitter events.

[0043] For the detection results obtained from the above three algorithms, a weighted calculation method is used to comprehensively judge whether abnormal jitter has occurred. Specifically, a weight coefficient W is assigned to each algorithm.GP ,W FP ,W FD , which correspond to the weights of the grayscale projection method, feature point matching method and frame difference method respectively, and then the weighted average value is calculated according to the following formula to obtain the abnormal jitter quantization value: S total =w GP ·S GP +W FP ·S FP +W FP ·S FP The final abnormal jitter quantization value S total The value is compared with the preset jitter threshold to determine whether abnormal jitter has occurred.

[0044] Further, as an embodiment of the present invention, in step S31, based on a preset grayscale projection method, grayscale projection curves of consecutive frame images are compared to obtain a first jitter intensity S GP ,include: Converting the image data into a grayscale image, and calculating a grayscale projection of the grayscale image in a horizontal direction and / or a vertical direction; The difference of the grayscale projection curves between the consecutive frame images is calculated and the first jitter intensity S is obtained by statistics. GP .

[0045] Specifically, first, the image data is converted into a grayscale image. This step converts each pixel of the color image into a corresponding grayscale value to simplify the subsequent processing process.

[0046] Next, the grayscale projection of the grayscale image in the horizontal and vertical directions is calculated respectively. Grayscale projection refers to the cumulative summation of the grayscale values ​​of the image pixels in a certain direction. The horizontal grayscale projection is to sum the grayscale values ​​of each row of the image to obtain the horizontal grayscale projection curve. The vertical grayscale projection is to sum the grayscale values ​​of each column of the image to obtain the vertical grayscale projection curve.

[0047] Calculate the difference of grayscale projection curves between consecutive frames. Assume that the horizontal grayscale projection of the current frame is P t , the horizontal grayscale projection of the previous frame is P t-1 , calculate the difference between the two: △P=|P t -P t-1 | Similarly, the grayscale projection difference in the vertical direction can also be calculated using a similar method, which will not be described in detail here.

[0048] The projection curve differences are statistically analyzed to obtain a first jitter intensity SGP , specifically, it can be calculated as follows: Cumulative difference value: The difference of the projection curve of each frame is accumulated and summed to obtain an overall first jitter intensity: Where n represents the number of data points in the projection curve; Normalization: In order to make S GP The values ​​have a certain scale and can be normalized by dividing the cumulative difference by a constant or the maximum value of the range.

[0049] In summary, the first jitter intensity S GP It is a quantitative value obtained by comparing the differences in the grayscale projection curves between consecutive frames and performing cumulative and statistical analysis. It reflects the overall displacement intensity of the image in the horizontal or vertical direction and is a key indicator generated by the grayscale projection method in jitter detection.

[0050] Further, as an implementation of the present invention, in step S32, based on a preset feature point matching algorithm, feature points in the continuous frame images are matched to obtain a second jitter strength S FP ,include: Detecting a set of feature points in the image data based on a preset feature extraction algorithm, and generating a feature descriptor for each feature point; Match feature points between consecutive frame images, and for each pair of matched feature points, calculate the displacement vector of the feature point between the previous frame and the current frame; The displacement vectors of all matching feature points are counted to obtain the second jitter strength S FP .

[0051] Specifically, first, a set of key feature points are extracted from the image. These feature points can be selected from areas with high contrast or significant structures in the image, such as corners, edges, etc. Common feature point extraction algorithms include SIFT, SURF, ORB, etc. A set of feature points is detected in each frame of the image using a specific algorithm, and the position, direction, scale, and other information of each feature point is recorded.

[0052] Next, each feature point is described to generate a feature descriptor, which contains key information about the pixel area around the feature point and is used to match the same feature point between different image frames.

[0053] Feature points are matched between consecutive frames. The goal of matching is to find the corresponding feature point pairs in the previous frame and the current frame. The feature matching algorithm can use the nearest neighbor search or other algorithms to compare the feature descriptors of the previous frame with the feature descriptors of the current frame to find the most similar feature point pairs.

[0054] For each pair of matched feature points, calculate their displacement vectors between the previous frame and the current frame. These displacement vectors reflect the movement of each feature point in the image: △P=P t -P t-1 Among them, P t and P t-1 Represent the positions of the corresponding feature points in the current frame and the previous frame respectively.

[0055] Perform statistical analysis on the displacement vectors of all matching feature points and calculate the overall second jitter intensity S FP , the specific calculation method can be: Among them, N is the matched feature point pair data, △P i is the displacement vector of the i-th pair of feature points.

[0056] In summary, the second jitter intensity S FP It is a quantitative value calculated by the feature point matching method, which reflects the displacement or change intensity of the key feature points in the image between consecutive frames. By statistically analyzing the displacement vector of the feature points, S FP It can accurately quantify the screen jitter and provide an important basis for subsequent abnormality detection.

[0057] Further, as an implementation of the present invention, in step S33, based on a preset frame difference method, pixel differences in consecutive frame images are compared to obtain a third jitter intensity S FD ,include: Calculate the pixel differences in consecutive frame images to obtain frame difference images; Performing threshold processing on the frame difference image to remove noise differences and obtain a valid frame difference image; The effective frame difference image is counted to obtain the third jitter intensity S FD .

[0058] Specifically, first, two consecutive frames of images (the previous frame I t-1 and the current frame I t ) for preprocessing

[0059] Calculate the current frame I t and the previous frame I t-1The pixel difference between them is used to obtain the frame difference image D t Each pixel value of the frame difference image represents the difference between the corresponding pixel of the current frame and the previous frame: D t (x,y)=|I t (x,y)-I t-1 (x,y)| Among them, D t (x, y) is the value of the pixel (x, y) in the frame difference image, I t (x,y) and I t-1 (x, y) are the grayscale values ​​of the corresponding pixels in the current frame and the previous frame respectively.

[0060] To remove noise, the frame difference image can be thresholded. Set a threshold T, and treat pixel differences less than the threshold as noise and ignore them, while pixels greater than the threshold are retained as valid changes:

[0061] The processed frame difference image D t 'Carry out statistics and calculate the overall third jitter intensity S FD The calculation methods include: Sum: Cumulatively sum the values ​​of all valid pixels in the frame difference image to obtain the third jitter intensity S FD : S FD =∑ x,y D t '(x,y); Normalize: Normalize the accumulated third jitter strength value by dividing it by the total number of pixels in the image to obtain a standardized jitter strength value.

[0062] In summary, S FD It is a quantitative value calculated by the frame difference method. It reflects the degree of pixel change between consecutive frames and is particularly suitable for detecting fast and sudden jitter. By calculating the change in the frame difference image, S FD The abnormal shaking of the vehicle screen can be effectively quantified and detected.

[0063] Further, as an embodiment of the present invention, refer to Figure 3 The non-intrusive screen shake detection method according to the embodiment of the present invention further includes: S4, when abnormal jitter is detected, the notification mechanism is triggered to report jitter-related information through a preset channel, wherein the jitter-related information includes video records during the abnormality, the precise time point of the abnormality, and / or the software and hardware log information of the vehicle screen.

[0064] Specifically, when abnormal jitter is detected, the notification mechanism is automatically triggered to quickly report key information through the preset channels, ensuring that test engineers can quickly respond to and handle abnormal situations. A variety of intelligent reporting channels can be integrated, including but not limited to email services that support Outlook and Exchange, as well as instant message reminder functions of mainstream office software such as DingTalk and Feishu.

[0065] Based on the specific needs of the test engineer and the support capabilities of the server, once a jitter anomaly is identified, a comprehensive report will be automatically integrated and sent to relevant personnel. The report includes video recording during the anomaly (with jitter frame identification), the exact time when the anomaly occurred, and the software and hardware log information of the vehicle screen (provided that a debug interface exists). This process is designed to efficiently identify, analyze, and solve problems.

[0066] In summary, the present invention provides a comprehensive and efficient vehicle computer abnormal jitter detection and automatic reporting solution.

[0067] First of all, the hardware cost required by the present invention is low, and it is completely non-intrusive detection, and only a simple combination of equipment is needed to meet the environment required for detection. At the same time, due to the involvement of motion sensors, the jitter of the recording equipment caused by the driving environment can be corrected, eliminating environmental interference to achieve a more accurate detection effect, while also taking into account more dynamic and static scene test environments.

[0068] Secondly, through the hybrid detection of the three algorithms, the present invention can detect abnormal shaking of the vehicle screen in all directions and at multiple angles, greatly improving the accuracy and reliability of the detection. This combined method can not only integrate the advantages of each algorithm, but also adapt to various complex driving and environmental conditions, thereby ensuring that abnormal shaking can be detected more accurately in most cases.

[0069] Finally, the present invention has a built-in intelligent reporting mechanism, which automatically reports key information to the preset notification channel when abnormal jitter is detected. This automated reporting mechanism includes email services that support Outlook and Exchange, as well as instant message reminder functions of mainstream office software such as DingTalk and Feishu. In this way, not only can the workload of manual inspection be greatly reduced, but also rapid abnormal response and processing can be achieved. Because the automatically reported information includes video records during the abnormality, the precise time point when the abnormality occurred, and the software and hardware log information of the vehicle screen. This detailed information can help engineers efficiently identify problems, analyze problems and find solutions, thereby greatly improving the efficiency of problem handling.

[0070] The embodiment of the present invention also discloses a non-intrusive detection system for screen shaking.

[0071] Reference Figure 4 A non-intrusive screen shake detection system includes a data acquisition device 1, a data processing device 2 and a data reporting device 3.

[0072] Reference Figure 4 and Figure 5 The data acquisition device 1 includes a collector 11 and a motion sensor 12 installed in the vehicle. The collector 11 is used to collect image data of the vehicle screen, and the motion sensor 12 is used to obtain the acquisition status data of the collector 11; specifically, the collector 11 can be a high-resolution industrial camera or camera, which is used to capture the image data of the vehicle screen in real time; the motion sensor 12 can include a gyroscope, accelerometer or other position sensor, which is installed on the collector 11 and is used to obtain the motion status data of the collector 11 during the driving process of the vehicle, such as displacement, rotation angle, acceleration, etc. These motion status data are recorded synchronously with the image data to provide a reference for subsequent image correction, ensuring that the system can effectively eliminate image distortion caused by the shaking of the collector 11, thereby improving the accuracy of abnormal shaking detection.

[0073] The data processing device 2 includes a correction module 21 and a detection module 22. The correction module 21 corrects the image data based on the acquisition status data to eliminate image distortion caused by the jitter of the collector 11. The detection module 22 is used to calculate the abnormal jitter quantization value of the corrected image data based on a preset abnormal jitter algorithm, and compare it with a preset jitter threshold to determine whether the screen has abnormal screen jitter.

[0074] The data processing device 2 is composed of a correction module 21 and a detection module 22. The correction module 21 performs real-time correction on the image data captured by the collector 11 based on the acquisition status data obtained from the motion sensor 12 to eliminate the image distortion caused by the jitter of the collector 11. The correction method can include image registration, geometric transformation, motion compensation and other technologies to ensure the stability and accuracy of the image. The detection module 22 uses a preset abnormal jitter algorithm (such as grayscale projection method, feature point matching method, frame difference method, etc.) to analyze the corrected image data, calculate the quantization value of the abnormal jitter, and compare it with the preset jitter threshold to determine whether the screen has abnormal jitter. The result of the detection module 22 can be used to trigger subsequent automatic reporting and abnormal handling processes.

[0075] The data reporting device 3 is used to trigger a notification mechanism when abnormal jitter is detected, and report jitter-related information through a preset channel, wherein the jitter-related information includes video records during the abnormality, the precise time point of the abnormality, and / or the software and hardware log information of the vehicle screen.

[0076] When the detection module 22 identifies abnormal screen jitter, the data reporting device 3 will automatically trigger the notification mechanism and report jitter-related information through preset channels (such as email, instant messaging applications, etc.). The reporting content includes video recording during the abnormal jitter, the precise time point of the abnormality, and the software and hardware log information of the vehicle screen (if there is a debugging interface). This information can be sent to relevant technicians or maintenance systems through the integrated network communication module to ensure rapid response and processing of screen jitter problems, thereby ensuring driving safety and the stability of the vehicle system.

[0077] The embodiment of the present invention also discloses a readable storage medium.

[0078] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the non-invasive detection method for screen jitter described in any one of the above embodiments. The computer-readable storage medium may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), and a software distribution medium, etc. The computer program includes a computer program code. The computer program code may be in source code form, object code form, an executable file, or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying a computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), and a software distribution medium, etc.

[0079] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-intrusive screen shake detection method, characterized in that: include: Collect image data from the vehicle screen and the collection status data from the collector; Correcting the image data based on the acquisition status data to eliminate image distortion caused by collector jitter; Based on a preset abnormal jitter algorithm, the abnormal jitter quantization value of the corrected image data is calculated and compared with a preset jitter threshold to determine whether the screen has abnormal screen jitter.

2. The non-intrusive screen shake detection method according to claim 1, characterized in that: The correcting the image data based on the acquisition status data comprises: For two consecutive frames of image data I t-1 Image frame and I t The image frames are aligned; Based on the result of the alignment process, t-1 The image frame is corrected to obtain the current I t Image frame.

3. The non-intrusive detection method for screen jitter according to claim 2, characterized in that: The image data is continuously processed by two frames of image I t-1 Image frame and I t The image frames are aligned, including: The transformation matrix T is generated based on the preset objective function calculation, and the transformation matrix T is used to transform the I t-1 Image frame conversion to I t Image frames, the objective function is used to characterize the alignment accuracy and / or transformation smoothness of two consecutive image frames; The change matrix T is obtained by minimizing the objective function.

4. The non-intrusive screen shake detection method according to claim 1, characterized in that: The calculating of the abnormal jitter quantization value of the corrected image data based on a preset abnormal jitter algorithm includes: Based on the preset grayscale projection method, the grayscale projection curves of the continuous frame images are compared to obtain the first jitter intensity S GP ; Based on the preset feature point matching algorithm, the feature points in the continuous frame images are matched to obtain the second jitter strength S FP ; Based on the preset frame difference method, the pixel differences in the consecutive frame images are compared to obtain the third jitter intensity S FD ; Based on a preset weighting algorithm, the first jitter strength S GP , the second jitter intensity S FP And the third jitter intensity S FD Perform weighted fusion to obtain the abnormal jitter quantization value.

5. The non-intrusive screen shake detection method according to claim 4, characterized in that: The grayscale projection method based on the preset grayscale projection method compares the grayscale projection curves of the continuous frame images to obtain the first jitter intensity S GP ,include: Converting the image data into a grayscale image, and calculating a grayscale projection of the grayscale image in a horizontal direction and / or a vertical direction; The difference of the grayscale projection curves between the consecutive frame images is calculated and the first jitter intensity S is obtained by statistics. GP .

6. The non-intrusive screen shake detection method according to claim 4, characterized in that: The feature points in the continuous frame images are matched based on the preset feature point matching algorithm to obtain the second jitter strength S FP ,include: Detecting a set of feature points in the image data based on a preset feature extraction algorithm, and generating a feature descriptor for each feature point; Match feature points between consecutive frame images, and for each pair of matched feature points, calculate the displacement vector of the feature point between the previous frame and the current frame; The displacement vectors of all matching feature points are counted to obtain the second jitter strength S FP .

7. The non-intrusive screen shake detection method according to claim 4, characterized in that: The pixel differences in the continuous frame images are compared based on the preset frame difference method to obtain the third jitter intensity S FD ,include: Calculate the pixel differences in consecutive frame images to obtain frame difference images; Performing threshold processing on the frame difference image to remove noise differences and obtain a valid frame difference image; The effective frame difference image is counted to obtain the third jitter intensity S FD .

8. The non-intrusive screen shake detection method according to claim 1, characterized in that: The method further comprises: When abnormal jitter is detected, the notification mechanism is triggered and jitter-related information is reported through a preset channel, wherein the jitter-related information includes video records during the abnormality, the precise time point when the abnormality occurred, and / or the software and hardware log information of the vehicle screen.

9. A non-intrusive screen shake detection system, characterized in that: include: The data acquisition device includes a collector and a motion sensor, wherein the collector is used to acquire image data of the vehicle-mounted screen, and the motion sensor is used to obtain acquisition status data of the collector; The data processing device includes a correction module and a detection module. The correction module corrects the image data based on the acquisition status data to eliminate image distortion caused by the jitter of the collector. The detection module is used to calculate the abnormal jitter quantization value of the corrected image data based on a preset abnormal jitter algorithm, and compare it with a preset jitter threshold to determine whether the screen has abnormal screen jitter.

10. The non-intrusive screen shake detection system according to claim 9, characterized in that: The system further comprises: The data reporting device is used to trigger the notification mechanism when abnormal jitter is detected, and report jitter-related information through a preset channel, wherein the jitter-related information includes video records during the abnormality, the precise time point when the abnormality occurred, and / or the software and hardware log information of the vehicle screen.

11. A readable storage medium, characterized in that: The readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, a non-intrusive method for detecting screen jitter according to any one of claims 1 to 8 is implemented.

Images