Visual compensation method, and electronic device and storage medium

Abstract

Disclosed in the present application are a visual compensation method, and an electronic device and a storage medium. The method comprises: acquiring motion attitude data of a vehicle, and video data collected by means of an in-vehicle camera; on the basis of the motion attitude data, correcting the video data; superimposing visual graphic information onto the corrected video data, so as to generate a compensated video, wherein the visual graphic information is used for indicating a motion state of the vehicle relative to a road; and performing visual compensation by means of the compensated video.

Description

Visual compensation methods, electronic devices and storage media

[0001] This application claims priority to Chinese Patent Application No. 202510470861.4, filed on April 15, 2025, entitled "Method for Visual Compensation, Electronic Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, and in particular to a method, electronic device and storage medium for visual compensation. Background Technology

[0003] Motion sickness (also known as motion sickness) is a feeling of dizziness caused by a conflict between the body's vestibular perception (such as the semicircular canals and otolith organs in the inner ear) and the motion information perceived by visual perception, which causes confusion in the brain.

[0004] Visual compensation can reduce the conflict between visual perception and vestibular perception, thus alleviating motion sickness in passengers. Summary of the Invention

[0005] This application provides a visual compensation method, electronic device, and storage medium for alleviating motion sickness in vehicle passengers. Specifically, it includes the following technical solutions.

[0006] In a first aspect, this application provides a method for visual compensation, the method comprising: acquiring vehicle motion posture data and video data captured by an onboard camera; correcting the video data based on the motion posture data; overlaying visual graphic information onto the corrected video data to generate a compensated video, wherein the visual graphic information is used to indicate the motion state of the vehicle relative to the road; and performing visual compensation through the compensated video.

[0007] In some possible implementations, the visual graphic information includes first graphic information indicating the current motion state of the vehicle relative to the road, and the method further includes: generating the first graphic information based on the motion posture data; and superimposing the visual graphic information into the corrected video data includes: superimposing the first graphic information into the corrected video data.

[0008] In some possible implementations, the visualized graphic information includes second graphic information for predicting the motion state of the vehicle relative to the road. The method further includes: acquiring navigation data of the vehicle; generating the second graphic information based on the navigation data; and superimposing the visualized graphic information into the corrected video data, which includes: superimposing the second graphic information into the corrected video data.

[0009] In some possible implementations, the correction includes jitter correction, the video data includes standard frames and jittered frames, and the correction of the video data based on the motion posture data includes: determining whether the vehicle is jittering based on the motion posture data; if the vehicle is jittering, determining a transformation matrix between the standard frames and the jittered frames based on the motion posture data; and performing jitter correction on the jittered frames based on the standard frames and the transformation matrix.

[0010] In some possible implementations, the motion posture data includes the vehicle speed information, the correction includes jitter correction, and the correction of the video data based on the motion posture data includes: if the speed of the vehicle indicated by the speed information exceeds a speed threshold, performing jitter correction on the video data corresponding to the speed information.

[0011] In some possible implementations, the visual compensation via the compensation video includes: playing the compensation video through the vehicle's in-vehicle display screen to perform visual compensation.

[0012] In some possible implementations, the method further includes: acquiring scene information related to the driving scenario of the vehicle; when playing the compensation video through the vehicle's in-vehicle display screen, the method further includes: adjusting the brightness of the compensation video according to the scene information; and adjusting the transparency and / or size of the visual graphic information in the compensation video according to the scene information.

[0013] In some possible implementations, when playing the compensation video through the vehicle's in-vehicle display screen, the method further includes: adjusting the viewing angle of the compensation video according to the position of the in-vehicle display screen.

[0014] In a second aspect, this application provides a visual compensation device, comprising: an acquisition module for acquiring vehicle motion posture data and video data captured by an onboard camera; a correction module for correcting the video data based on the motion posture data; a graphics overlay module for overlaying visual graphics information onto the corrected video data to generate a compensated video, wherein the visual graphics information is used to indicate the vehicle's motion state relative to the road; and a compensation module for performing visual compensation using the compensated video.

[0015] In some possible implementations, the visual graphic information includes first graphic information indicating the current motion state of the vehicle relative to the road, and the graphic overlay module is used to generate the first graphic information based on the motion posture data and overlay the first graphic information onto the corrected video data.

[0016] In some possible implementations, the visualization graphic information includes second graphic information for predicting the motion state of the vehicle relative to the road, and the acquisition module is further used to acquire the navigation data of the vehicle; the graphic overlay module is used to generate the second graphic information based on the navigation data and overlay the second graphic information onto the corrected video data.

[0017] In some possible implementations, the correction includes jitter correction, the video data includes standard frames and jitter frames, and the correction module is used to determine whether the vehicle is jittering based on the motion posture data; if the vehicle is jittering, a transformation matrix between the standard frames and the jitter frames is determined based on the motion posture data; and jitter correction is performed on the jitter frames based on the standard frames and the transformation matrix.

[0018] In some possible implementations, the motion posture data includes the vehicle speed information, the correction includes jitter correction, and the correction module is used to perform jitter correction on the video data corresponding to the vehicle speed information if the speed of the vehicle indicated by the vehicle speed information exceeds a speed threshold.

[0019] In some possible implementations, the compensation module is used to play the compensation video through the vehicle's in-vehicle display screen to perform visual compensation.

[0020] In some possible implementations, the acquisition module is further configured to acquire scene information related to the driving scene of the vehicle; the compensation module is further configured to adjust the brightness of the compensation video according to the scene information; and adjust the transparency and / or size of the visual graphic information in the compensation video according to the scene information.

[0021] In some possible implementations, the compensation module is also used to adjust the viewing angle of the compensated video according to the position of the vehicle display screen.

[0022] In a third aspect, this application provides an electronic device for vision compensation, comprising: a memory storing at least one program instruction for vision compensation; and a processor, wherein when the program instruction is executed by the processor, it causes a vehicle to implement the method of the first aspect of this application or any possible implementation thereof.

[0023] In a fourth aspect, this application provides a computer program (product) comprising computer program / instructions, which are executed by a processor to cause a vehicle to implement the method of the first aspect of this application or any possible implementation thereof.

[0024] In a fifth aspect, this application provides a computer-readable storage medium having stored thereon program instructions for visual compensation, which, when executed by one or more processors, cause a vehicle to implement the method of the first aspect or any possible implementation of the first aspect.

[0025] The beneficial effects of the technical solution provided in this application include at least the following:

[0026] The technical solution provided in this application, on the one hand, can visually compensate for the visual perception of passengers inside the vehicle by generating compensated videos based on video data collected by the vehicle-mounted camera, enabling passengers to more intuitively perceive the vehicle's motion relative to the road and avoid motion sickness; on the other hand, it corrects the video data using vehicle motion posture data and guides passengers through visual graphic information superimposed on the video data, allowing passengers to maintain consistency between visual perception and vestibular perception, which further helps alleviate motion sickness. The above methods not only eliminate the need for additional vehicle hardware but also significantly improve the comfort and convenience of the passenger's travel experience. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 is a schematic diagram of the implementation scenario provided in the embodiments of this application;

[0029] Figure 2 is a flowchart of the visual compensation method provided in an embodiment of this application;

[0030] Figure 3 is a schematic diagram of the structure of the visual compensation device provided in an embodiment of this application;

[0031] Figure 4 is a schematic diagram of the structure of an electronic device for visual compensation provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] Figure 1 is a schematic diagram of an implementation scenario provided by an embodiment of this application. Referring to Figure 1, the implementation scenario provided by an embodiment of this application may include a vehicle 11 and an in-vehicle display screen 12.

[0035] The vehicle 11 is equipped with an on-board control unit, which can generate compensation video based on the motion attitude data of the vehicle 11 and the video data collected by the on-board camera in the vehicle 11, so that the visual perception of passengers in the vehicle can be compensated through the compensation video.

[0036] The in-vehicle display screen 12 can be connected to the in-vehicle control unit installed in the vehicle 11 via wired or wireless means, so that the in-vehicle display screen 12 can play compensated videos under the control of the in-vehicle control unit.

[0037] Optionally, the on-board control unit installed in vehicle 11 can be a terminal controller installed in the vehicle, or it can be a server, a server cluster consisting of multiple servers, or a cloud computing service center.

[0038] Those skilled in the art should understand that the above-described vehicle control unit and vehicle display screen are merely examples. Other existing or future vehicle control units and vehicle display screens that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0039] When a vehicle is in motion, the vehicle's cabin or other structural components may obstruct the passengers' view, causing a conflict between the passengers' visual perception of the vehicle's motion and their vestibular perception (for example, the visual perception is of a stationary interior environment, while the vestibular perception is of the vehicle's actual motion), thus causing the passengers to experience motion sickness.

[0040] In related technologies, methods to alleviate motion sickness include medication, motion sickness seats, motion sickness-proof windows, and motion sickness glasses. However, medication may have side effects, motion sickness seats and windows significantly increase the vehicle's hardware costs, and motion sickness glasses are less convenient and negatively impact the passenger experience.

[0041] To address the aforementioned issues, this application provides a visual compensation method that can compensate for the visual perception of passengers inside the vehicle through compensation video. This allows passengers to more intuitively determine the vehicle's motion state through visual perception, thereby maintaining consistency between visual perception and vestibular perception, avoiding motion sickness, and offering high convenience and low cost.

[0042] Figure 2 is a flowchart of a vision compensation method provided in an embodiment of this application. This method can be executed, for example, by an onboard control unit mounted in a vehicle, and this application makes no limitations in this regard. Referring to Figure 2, the vision compensation method provided in an embodiment of this application may include the following steps.

[0043] Step S210: Obtain vehicle motion attitude data and video data collected by the vehicle-mounted camera.

[0044] For example, vehicle motion attitude data is used, but not limited to, to indicate any data related to the vehicle's motion, such as data indicating the vehicle's steering wheel direction, steering wheel angle, vehicle speed, brake pedal depth, and / or accelerator pedal depth. This application makes no limitations in this regard. In some embodiments, the method for acquiring vehicle motion attitude data includes, for example, collecting vehicle motion attitude data using one or more on-board sensors mounted in the vehicle. These on-board sensors include, but are not limited to, gyroscopes, accelerometers, accelerator pedal opening sensors, brake pedal opening sensors, and steering wheel angle sensors.

[0045] The video data collected by the vehicle-mounted camera can be used, but is not limited to, to indicate the relative position and motion of the vehicle and its surrounding environment. The vehicle-mounted camera includes, for example, a vehicle's dashcam or any other device capable of collecting video data. The surrounding environment includes, for example, the sum of all natural and man-made factors around the vehicle during its movement, such as the road on which the vehicle is located, other vehicles around the vehicle, pedestrians, etc. This application makes no limitations in this regard.

[0046] Step S220: Correct the video data based on the motion posture data.

[0047] Considering that in real-world applications, the vehicle's motion can affect the stability and continuity of video data, potentially leading to video frames that exacerbate motion sickness in passengers. For example, when the vehicle experiences bumps, the stability of the video data deteriorates, potentially resulting in shaky frames. These shaky frames indicate a discrepancy between the relative position and motion of the vehicle and its surroundings and the actual relative position and motion. Similarly, at higher speeds, the continuity of the video data deteriorates, potentially resulting in jumpy frames. This means there's a disconnect between the relative position and motion of the vehicle and its surroundings at two different moments indicated by adjacent video frames. Therefore, to ensure the compensating video effectively compensates for passengers' visual perception, the video data captured by the vehicle's cameras can be corrected. For instance, when the vehicle experiences bumps, to prevent shaky video frames from exacerbating motion sickness in passengers, shake correction can be applied to the video data captured by the vehicle's cameras. In some embodiments, the correction includes jitter correction, and the video data includes, for example, standard frames and jitter frames. The correction of the video data based on motion posture data includes, for example, determining whether the vehicle is jittering based on the motion posture data; if the vehicle is jittering, determining a transformation matrix between the standard frames and jitter frames based on the motion posture data; and performing jitter correction on the jitter frames based on the standard frames and the transformation matrix.

[0048] Standard frames include, for example, video frames in which the indicated relative position and motion relationship between the vehicle and its surrounding environment is consistent with the actual relative position and motion relationship between the vehicle and its surrounding environment. Jittery frames include, for example, video frames in which the indicated relative position and motion relationship between the vehicle and its surrounding environment deviates from the actual relative position and motion relationship between the vehicle and its surrounding environment.

[0049] The transformation matrix is ​​used, but is not limited to, indicating, the changes in the relative position and motion between the vehicle and its surrounding environment between a first moment corresponding to a standard frame and a second moment corresponding to a jittery frame. The transformation matrix also indicates the quantification of the geometric changes between the standard frame and the jittery frame, used, but not limited to, describing, the spatial attitude changes caused by vehicle jitter between the first moment corresponding to the standard frame and the second moment corresponding to the jittery frame, and the resulting image distortion. Exemplarily, the transformation matrix may include: a translation mapping matrix, used, but not limited to, correcting positional offset distortion in the jittery frame; a scaling mapping matrix, used, but not limited to, correcting image scaling distortion in the jittery frame; and a tilt mapping matrix, used, but not limited to, correcting shearing or tilt distortion in the jittery frame.

[0050] In some embodiments, when the video data is acquired by a dashcam in a vehicle, the method for determining the transformation matrix includes, for example, determining the spatial attitude change of the dashcam between a first time moment and a second time moment based on motion attitude data; and determining the transformation matrix based on the spatial attitude change. The spatial attitude change of the dashcam indicates the translation, rotation, tilt, and other changes of the dashcam in space during the period from the first time moment to the second time moment. By determining the spatial attitude change of the dashcam through motion attitude data, the physical spatial attitude change of the dashcam is transformed into a mapping relationship between image pixels between standard frames and jittery frames, i.e., the transformation matrix. This embodiment of the application can accurately compensate for image distortion caused by jitter and improve the stability of the image in the video data.

[0051] In some embodiments, the motion posture data includes, for example, the longitudinal displacement data of the vehicle. A method for determining whether the vehicle is vibrating based on the motion posture data includes, for example, determining that the vehicle is vibrating if the rate of change of the longitudinal displacement of the vehicle within a preset time period is greater than a first threshold. The longitudinal displacement data is, for example, the displacement data of the vehicle along the horizontal plane. The value of the first threshold can be adjusted according to actual application conditions, and this application does not impose any limitations in this regard.

[0052] When the vehicle is traveling at high speeds, to prevent jerky video footage from exacerbating motion sickness for passengers, jerky correction can be applied to the video data captured by the vehicle's camera. In some embodiments, motion posture data includes, for example, vehicle speed information, and correction includes jerky correction. Correcting the video data based on the motion posture data includes, for example, correcting the video data corresponding to the speed information if the vehicle speed indicated by the speed information exceeds a speed threshold.

[0053] The speed threshold value can be adjusted according to the actual application, and this application does not impose any restrictions in this regard. Methods for correcting jitter in video data corresponding to vehicle speed information include, for example, generating a transition frame between two adjacent video frames with jitter through frame interpolation. The transition frame is, for example, a video frame that reduces the sense of disjointedness between the relative positional relationship and relative motion relationship between the vehicle and its surrounding environment indicated by two adjacent video frames.

[0054] Step S230: Overlay visual graphic information onto the corrected video data to generate a compensated video. The visual graphic information is used to indicate the vehicle's motion relative to the road.

[0055] In some embodiments, the method of overlaying visual graphic information onto the corrected video data may include, for example, overlaying the visual graphic information onto the corrected video data using video editing software (e.g., Adobe Premiere Pro, Final Cut Pro, DaVinci Resolve, etc.) or 3D engine tools (e.g., Unreal Engine, Unity, Blender, etc.).

[0056] As mentioned earlier, video data collected by in-vehicle cameras can indicate the relative position and motion of the vehicle with its surroundings. Considering that complex relative position and motion relationships between the vehicle and its environment can increase the visual burden on passengers, making it difficult for them to intuitively determine the vehicle's motion state visually—for example, the presence of multiple other vehicles with varying relative motions can further burden their visual perception—this application simplifies the complex relative position and motion relationships between the vehicle and its surroundings to the vehicle's motion state relative to the road by providing visual graphical information that indicates the vehicle's motion state relative to the road. This reduces the visual burden on passengers, enhances their intuitive understanding of the vehicle's motion state, and helps alleviate motion sickness.

[0057] The vehicle's motion state relative to the road includes, for example, the vehicle's current motion state relative to the road, i.e., the vehicle's real-time motion state relative to the road. The visualized graphic information includes, for example, first graphic information indicating the vehicle's current motion state relative to the road. The method provided in this application includes: generating first graphic information based on motion posture data. In some embodiments, when the visualized graphic information is the first graphic information, the visualized graphic information is superimposed on the corrected video data, for example, including: superimposing the first graphic information onto the corrected video data.

[0058] For example, the first graphic information may include, for instance, a steering wheel graphic indicating the steering wheel direction and angle of the vehicle, an instrument panel graphic indicating the vehicle speed, a first straight arrow graphic indicating the vehicle acceleration, a first curved arrow graphic indicating the vehicle turning, accelerator pedal graphics and brake pedal graphics indicating the vehicle pedal depth, a level graphic indicating the vehicle's left and right tilt angle and front and rear pitch angle, etc., and this application does not impose any limitations in this regard.

[0059] The vehicle's motion state relative to the road includes, for example, the future motion state of the vehicle relative to the road indicated by the vehicle's navigation information. Visualized graphical information includes, for example, second graphical information used to help passengers inside the vehicle predict the vehicle's motion state relative to the road. The method provided in this application includes: acquiring vehicle navigation data; and generating second graphical information based on the navigation data. In some embodiments, when the visualized graphical information is the second graphical information, the visualized graphical information is superimposed on the corrected video data, for example, by superimposing the second graphical information onto the corrected video data.

[0060] The vehicle's navigation data can be obtained through GNSS (Compass Navigation Satellite System) or GPS (Global Positioning System), and this application does not impose any restrictions in this regard.

[0061] The second graphic information may include, for example, a second straight arrow graphic to help passengers inside the vehicle anticipate that the vehicle is going straight, or a second curved arrow graphic to help passengers inside the vehicle anticipate that the vehicle is turning, etc. This application does not impose any limitations in this regard.

[0062] Alternatively, the second graphic information may include motion graphic information to help passengers inside the vehicle anticipate the vehicle's movement and road condition graphic information to help passengers anticipate that the vehicle will enter a special road section. For example, motion graphic information may include a second straight arrow graphic and a countdown graphic indicating that the vehicle will go straight after a specified time threshold, and a second curved arrow graphic and a countdown graphic indicating that the vehicle will turn after a specified time threshold; road condition graphic information may include a roundabout sign graphic and a countdown graphic indicating that the vehicle will enter a roundabout after a specified time threshold, and a tunnel sign graphic and a countdown graphic indicating that the vehicle will enter a tunnel after a specified time threshold. This application does not impose any restrictions in this regard. The specified time thresholds corresponding to different second graphic information may be the same or different, and the value of the specified time threshold may be adjusted according to the actual application scenario. This application does not impose any restrictions in this regard.

[0063] Step S240: Perform visual compensation using the compensation video.

[0064] Optionally, the compensation video is used, but not limited to, to convert the vehicle's motion state into visual information to compensate for the discrepancy between the passenger's visual perception of the vehicle's motion state and the vehicle's actual motion state when the passenger's line of sight is obstructed by the passenger compartment or other structural components of the vehicle. This ensures that the passenger's visual perception is consistent with their vestibular perception, thus avoiding motion sickness.

[0065] In some embodiments, visual perception of passengers inside the vehicle is compensated by compensating video, such as by playing a compensation video on the vehicle's in-vehicle display screen to perform visual compensation.

[0066] The in-vehicle display screen may include one or more. When a vehicle has multiple in-vehicle display screens, the multiple in-vehicle display screens may be set in different locations inside the vehicle to compensate for the visual perception of passengers in different locations inside the vehicle.

[0067] Considering that passengers in different positions within the vehicle may have different visual perceptions of the vehicle's motion, the method provided in this application embodiment, when playing compensation video through the vehicle's in-vehicle display screen, includes, for example, adjusting the viewing angle of the compensation video according to the position of the in-vehicle display screen, so that the viewing angle of the compensation video matches the direction of head movement of the passengers inside the vehicle.

[0068] It also takes into account that when the driving scene changes, the surrounding environment may affect the visual perception of passengers inside the vehicle, thereby affecting the compensation effect of the compensation video on the visual perception of passengers inside the vehicle. For example, when the vehicle moves from a low-light environment to a high-light environment, the visual perception of passengers inside the vehicle by the compensation video will change.

[0069] In view of this, the method provided in the embodiments of this application includes, for example,: acquiring scene information related to the vehicle's driving scene; adjusting the brightness of the compensation video according to the scene information when playing the compensation video through the vehicle's in-vehicle display screen; and adjusting the transparency and / or size of the visual graphic information in the compensation video according to the scene information. For example, when the light intensity of the environment around the vehicle indicated by the scene information increases, the brightness of the compensation video is increased, and the contrast of the visual graphic information in the compensation video is increased; or, when the light intensity of the environment around the vehicle indicated by the scene information increases, the brightness of the compensation video is increased, and the size of the visual graphic information in the compensation video is increased; or, when the light intensity of the environment around the vehicle indicated by the scene information increases, the brightness of the compensation video is increased, and the contrast of the visual graphic information in the compensation video is increased and the size of the visual graphic information is increased.

[0070] The technical solution provided in this application, on the one hand, can visually compensate for the visual perception of passengers inside the vehicle by generating compensated videos based on video data collected by the vehicle-mounted camera, enabling passengers to more intuitively perceive the vehicle's motion relative to the road and avoid motion sickness; on the other hand, it corrects the video data using vehicle motion posture data and guides passengers through visual graphic information superimposed on the video data, allowing passengers to maintain consistency between visual perception and vestibular perception, which further helps alleviate motion sickness. The above methods not only eliminate the need for additional vehicle hardware but also significantly improve the comfort and convenience of the passenger's travel experience.

[0071] In some other possible implementations, this application also provides a visual compensation device. Figure 3 is a schematic diagram of the structure of the visual compensation device provided in an embodiment of this application. Referring to Figure 3, the visual compensation device provided in an embodiment of this application includes the following modules.

[0072] The acquisition module 310 is used to acquire vehicle motion posture data and video data collected by the vehicle-mounted camera.

[0073] The correction module 320 is used to correct video data based on motion posture data.

[0074] The graphics overlay module 330 is used to overlay visual graphic information onto the corrected video data to generate a compensated video. The visual graphic information is used to indicate the vehicle's motion relative to the road.

[0075] The compensation module 340 is used for visual compensation through the compensation video.

[0076] In some possible implementations, the visual graphic information includes first graphic information for indicating the current motion state of the vehicle relative to the road, and the graphic overlay module 330 is used to generate the first graphic information based on the motion posture data and overlay the first graphic information onto the corrected video data.

[0077] In some possible implementations, the visualization graphic information includes second graphic information for predicting the motion state of the vehicle relative to the road, and the acquisition module 310 is also used to acquire the vehicle's navigation data; the graphic overlay module 330 is used to generate the second graphic information based on the navigation data and overlay the second graphic information onto the corrected video data.

[0078] In some possible implementations, the correction includes jitter correction, the video data includes standard frames and jitter frames, and the correction module 320 is used to determine whether the vehicle is jittering based on the motion posture data; if the vehicle is jittering, a transformation matrix between the standard frames and jitter frames is determined based on the motion posture data; and jitter correction is performed on the jitter frames based on the standard frames and the transformation matrix.

[0079] In some possible implementations, the motion posture data includes vehicle speed information, and the correction includes jitter correction. The correction module 320 is used to perform jitter correction on the video data corresponding to the vehicle speed information if the vehicle speed indicated by the vehicle speed information exceeds a speed threshold.

[0080] In some possible implementations, the compensation module 340 is used to play a compensation video through the vehicle's in-vehicle display screen to perform visual compensation.

[0081] In some possible implementations, the acquisition module 310 is further configured to acquire scene information related to the vehicle's driving scene; the compensation module 340 is further configured to adjust the brightness of the compensation video according to the scene information; and to adjust the transparency and / or size of the visual graphic information in the compensation video according to the scene information.

[0082] In some possible implementations, the compensation module 340 is also used to adjust the viewing angle of the compensation video according to the position of the vehicle display screen.

[0083] It should be understood that the visual compensation device shown in Figure 3 above is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The visual compensation device and the visual compensation method embodiment provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the visual compensation method embodiment.

[0084] In some other possible implementations, this application also provides an electronic device for visual compensation. Figure 4 is a schematic diagram of the structure of the electronic device for visual compensation provided in an embodiment of this application. Referring to Figure 4, the electronic device for visual compensation provided in an embodiment of this application includes:

[0085] The memory 410 stores at least one program instruction for visual compensation.

[0086] When the processor 420 executes the aforementioned program instructions, it causes the vehicle to implement the steps of the method and its various embodiments described above in conjunction with FIG2. Depending on the implementation, the processor 420 may be one or more types of processors, including but not limited to DSP (digital signal processor), ASIC (application specific integrated circuit), FPGA (field-programmable gate array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and the number of such processors may be determined according to actual needs.

[0087] In some other possible implementations, this application also provides a computer program (product) including computer program / instructions that are executed by a processor to cause a vehicle to perform the steps of the method and its various embodiments described above in conjunction with FIG2.

[0088] In some other possible implementations, this application also provides a computer-readable storage medium storing program instructions for visual compensation, which, when executed by one or more processors, cause a vehicle to perform the steps of the method and its various embodiments described above in conjunction with FIG2. The computer-readable storage medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0089] It should also be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0090] The term "and / or" in the embodiments of this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0091] The above description is only for the purpose of enabling those skilled in the art to understand the technical solution of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for visual compensation, wherein, The method includes: Acquire vehicle motion attitude data and video data captured by onboard cameras; The video data is corrected based on the motion posture data; Visual graphic information is overlaid on the corrected video data to generate a compensated video, wherein the visual graphic information is used to indicate the motion state of the vehicle relative to the road; Visual compensation is performed using the compensated video.

2. The method according to claim 1, wherein, The visualized graphical information includes first graphical information indicating the current motion state of the vehicle relative to the road, and the method further includes: The first graphic information is generated based on the motion posture data; The process of overlaying visual graphic information onto the corrected video data includes: The first graphic information is superimposed onto the corrected video data.

3. The method according to claim 1, wherein, The visualized graphical information includes second graphical information for predicting the vehicle's motion state relative to the road, and the method further includes: Obtain the navigation data of the vehicle; The second graphic information is generated based on the navigation data; The process of overlaying visual graphic information onto the corrected video data includes: The second graphic information is superimposed onto the corrected video data.

4. The method according to claim 1, wherein, The correction includes jitter correction, the video data includes standard frames and jittery frames, and the correction of the video data based on the motion posture data includes: Determine whether the vehicle is shaking based on the motion posture data; If the vehicle shakes, a transformation matrix between the standard frame and the shaking frame is determined based on the motion posture data. The jitter correction is performed on the jittered frame based on the standard frame and the transformation matrix.

5. The method according to claim 1, wherein, The motion posture data includes the vehicle speed information, the correction includes jitter correction, and the correction of the video data based on the motion posture data includes: If the speed of the vehicle indicated by the vehicle speed information exceeds the speed threshold, the jitter correction is performed on the video data corresponding to the vehicle speed information.

6. The method according to any one of claims 1-5, wherein, The visual compensation performed using the compensated video includes: The compensation video is played on the vehicle's in-vehicle display screen to perform visual compensation.

7. The method according to claim 6, wherein, The method further includes: Obtain scene information related to the driving scenario of the vehicle; When playing the compensated video through the vehicle's in-vehicle display screen, the method further includes: The brightness of the compensated video is adjusted based on the scene information; and, The transparency and / or size of the visual graphic information in the compensation video are adjusted based on the scene information.

8. The method according to claim 6, wherein, When playing the compensated video through the vehicle's in-vehicle display screen, the method further includes: The angle of the compensated video is adjusted according to the position of the vehicle-mounted display screen.

9. A visual compensation device, wherein, The device includes: The acquisition module is used to acquire vehicle motion posture data and video data captured by the vehicle-mounted camera; A correction module is used to correct the video data based on the motion posture data; A graphic overlay module is used to overlay visual graphic information onto the corrected video data to generate a compensated video, wherein the visual graphic information is used to indicate the motion state of the vehicle relative to the road. The compensation module is used to perform visual compensation through the compensation video.

10. The apparatus according to claim 9, wherein, The visual graphic information includes first graphic information used to indicate the current motion state of the vehicle relative to the road; The graphics overlay module is used to generate the first graphics information based on the motion posture data, and to overlay the first graphics information onto the corrected video data.

11. The apparatus according to claim 9, wherein, The visual graphic information includes second graphic information used to predict the motion state of the vehicle relative to the road; The acquisition module is also used to acquire the vehicle's navigation data; The graphic overlay module is used to generate the second graphic information based on the navigation data, and to overlay the second graphic information onto the corrected video data.

12. The apparatus according to claim 9, wherein, The correction includes jitter correction, and the video data includes standard frames and jittery frames; The correction module is used to determine whether the vehicle is shaking based on the motion posture data; if the vehicle is shaking, it determines the transformation matrix between the standard frame and the shaking frame based on the motion posture data; and performs the shaking correction on the shaking frame based on the standard frame and the transformation matrix.

13. The apparatus according to claim 9, wherein, The motion posture data includes the vehicle speed information, and the correction includes bounce correction; The correction module is used to perform jitter correction on the video data corresponding to the vehicle speed information if the speed of the vehicle indicated by the vehicle speed information exceeds a speed threshold.

14. The apparatus according to any one of claims 9-13, wherein, The compensation module is used to play the compensation video through the vehicle's in-vehicle display screen to perform visual compensation.

15. The apparatus according to claim 14, wherein, The acquisition module is also used to acquire scene information related to the driving scene of the vehicle; The compensation module is also used to adjust the brightness of the compensated video according to the scene information; In addition, the transparency and / or size of the visual graphic information in the compensated video are adjusted according to the scene information.

16. The apparatus according to claim 14, wherein, The compensation module is also used to adjust the viewing angle of the compensation video according to the position of the vehicle display screen.

17. A computer-readable storage medium, wherein, It stores program instructions for visual compensation, which, when executed by one or more processors, cause the vehicle to perform the method described in any one of claims 1-8.

18. An electronic device, wherein, include: The memory contains program instructions for visual compensation. as well as A processor, when the program instructions are executed by the processor, causes the vehicle to perform the method described in any one of claims 1-8.

19. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, causes the vehicle to perform the method described in any one of claims 1-8.

Images