Visual field display method and device based on vehicle-mounted screen and vehicle-mounted screen

Abstract

The invention provides a visual field display method and device based on a vehicle-mounted screen and the vehicle-mounted screen, and is applied to the technical field of automobiles, and the method comprises the following steps: in response to a visual field display instruction of a rear-row passenger, obtaining first seat information of a rear-row seat, second seat information of a front-row seat and first face information of the rear-row passenger; a video in front of a vehicle collected by a camera arranged in front of the vehicle is converted into a real-time image frame. And by utilizing the first seat information, the second seat information and the first face information, picture information that the front-row seats shield the view of the rear-row passengers is cut out from the real-time image frame. The picture information comprises the first shielding picture and a reduction scale corresponding to the first shielding picture. The first shielding picture is adjusted according to the reduction scale, the adjusted first shielding picture is displayed to the rear-row passengers through the vehicle-mounted screen located on the front-row seats, the road conditions in front of and behind the vehicle can be displayed in real time through the vehicle backseat screen, and the riding safety and comfort of the rear-row passengers are improved.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a method and device for displaying a field of view based on a vehicle-mounted screen, and a vehicle-mounted screen. Background Art

[0002] In-car screens (also called rear-seat entertainment systems or rear-seat entertainment systems, RSE) are common features in modern cars, especially in high-end and luxury cars. These screens are usually installed behind the headrests of the front seats or in the roof lining, and are mainly used to play videos, music, and other multimedia content to provide entertainment experience for rear passengers. However, these headrest screens usually do not have the function of displaying the road conditions in front and behind the vehicle, resulting in the rear passengers being unable to fully observe the road conditions in front and behind due to the obstruction of the seats, affecting the safety and comfort of the ride.

[0003] Therefore, there is an urgent need for a field of view display method based on a vehicle-mounted screen, which is used to display the road conditions in front and behind the vehicle in real time through the vehicle's rear seat screen, thereby improving the safety and comfort of rear passengers. Summary of the invention

[0004] The purpose of this application is to provide a vehicle-mounted screen-based field of view display method, device and vehicle-mounted screen, so as to display the road conditions in front and behind the vehicle in real time through the vehicle's rear seat screen, thereby improving the safety and comfort of rear passengers.

[0005] In a first aspect, an embodiment of the present application provides a method for displaying a field of view based on a vehicle screen, which is applied to a vehicle display system. The vehicle display system includes a vehicle screen arranged on the back of a front seat for viewing by a rear passenger sitting in a rear seat. The method includes: in response to a rear passenger's field of view display instruction, obtaining first seat information of the rear seat, second seat information of the front seat, and first facial information of the rear passenger. The rear seat is a seat for the rear passenger to sit on. The front seat is a seat that blocks the field of view of the rear passenger. The video in front of the vehicle collected by a camera deployed in front of the vehicle is converted into a real-time image frame. Using the first seat information, the second seat information, and the first facial information, the picture information of the front seat blocking the field of view of the rear passenger is cropped from the real-time image frame. The picture information includes a first blocking picture and a reduction ratio corresponding to the first blocking picture. The first blocking picture is adjusted according to the reduction ratio, and the adjusted first blocking picture is displayed to the rear passenger through the vehicle screen located on the front seat.

[0006] The method for displaying the visual field based on the vehicle screen provided in the embodiment of the present application can display the visual field blocked by the front seats, so that the rear passengers can obtain a more comprehensive visual field in front, thereby improving the riding experience, especially during long-distance travel, and reducing the discomfort of the rear passengers. At the same time, the rear passengers can view the road conditions ahead through the vehicle screen, which helps to respond in time in an emergency and increases the safety of riding.

[0007] In a possible implementation, the first seat information includes first size information and first state information. The second seat information includes second size information and second state information. Obtaining the first seat information of the rear seat, the second seat information of the front seat, and the first facial information of the rear passenger includes: reading the first size information and the second size information stored in the control system from the vehicle's control system. Obtaining the first state information and the second state information using an onboard screen. The first state information includes a first distance from the front windshield of the vehicle and a first tilt angle. The second state information includes a second distance from the front windshield and a second tilt angle. Obtaining the first facial information using an onboard screen.

[0008] In a possible implementation, the first facial information includes the eye position. The first seat information, the second seat information and the first facial information are used to crop a first blocking picture in which the front seat blocks the field of view of the rear passengers from the real-time image frame, including: determining the line of sight of the rear passengers using the first distance, the first tilt angle, the second distance, the second tilt angle and the eye position. The viewing picture of the rear passengers is determined using the first distance, the first tilt angle, the second distance, the second tilt angle and the line of sight. The viewing picture is the picture that the rear passengers see in the line of sight and is not blocked by the front seats. The first blocking picture is cropped from the real-time image frame using the viewing picture, and the reduction ratio is determined based on the viewing picture, the real-time image frame and the first blocking picture.

[0009] In a possible implementation, the second seat information also includes: body size information of the front passenger, the front passenger is a passenger sitting in the front seat. The first obstruction picture includes a second obstruction picture and a third obstruction picture, the second obstruction picture is a picture in which the field of view of the rear passenger is obstructed by the front seat, and the third obstruction picture is a picture in which the rear passenger is obstructed by the front passenger. Using the viewing picture to crop the first obstruction picture from the real-time image frame includes: using the viewing picture to crop the second obstruction picture and the third obstruction picture from the real-time image frame. Combining the second obstruction picture and the third obstruction picture into the first obstruction picture.

[0010] In a possible implementation, the first facial information also includes an eye gaze position. The first obstructed image includes a highlighted target. The method further includes: obtaining historical eye gaze positions corresponding to a plurality of historical image frames before the real-time image frame within a preset time period. If the number of historical image frames corresponding to the same historical eye gaze position reaches a number threshold, adjusting the focal length of a camera deployed in front of the vehicle so that the target of the first obstructed image corresponding to the gaze position is the highlighted target.

[0011] In a possible implementation, the method for displaying the visual field based on the vehicle-mounted screen provided in the embodiment of the present application further includes: when the highlighted target is a building, obtaining detailed information of the building, and displaying the detailed information on the first obstruction screen.

[0012] In a possible implementation, the first facial information includes micro-expression information. The field of view display method based on the vehicle screen provided in the embodiment of the present application further includes: in response to the virtual display task submitted by the user on the vehicle screen, displaying the virtual target on the first occlusion screen. Based on the micro-expression information and detailed information, determining the feedback information of the virtual target, and updating the virtual display task based on the feedback information.

[0013] In a possible implementation, the field of view display method based on the vehicle screen provided in the embodiment of the present application also includes: determining the reference object in front of the vehicle and the reference object position set in each historical image frame based on multiple historical image frames before the real-time image frame within a preset time period. Based on the reference object position set, determine the relative movement trend between the reference object and the vehicle. The relative movement trend includes relative position and relative speed. Based on the relative position and relative speed, predict the real-time position of the reference object in the real-time image frame. If the distance difference between the real-time position and the current position of the vehicle reaches a distance threshold, generate a warning message. The warning message is used to remind rear passengers that the vehicle is in danger.

[0014] In a possible implementation, the method for displaying the visual field based on the vehicle screen provided in the embodiment of the present application further includes: when the movement of the rear passenger is detected, obtaining the second facial information of the rear passenger, using the second facial information to correct the image information, and displaying the corrected image information to the rear passenger through the vehicle screen.

[0015] In a possible implementation manner, the field of view display method based on the vehicle-mounted screen provided in the embodiment of the present application further includes: generating a field of view display instruction when it is detected that the line of sight of the rear passenger is looking at the vehicle-mounted screen.

[0016] In the second aspect, an embodiment of the present application provides a field of view display device based on a vehicle-mounted screen, which is applied to a vehicle display system. The vehicle display system includes a vehicle-mounted screen arranged on the back of the front seats for viewing by rear passengers sitting in the rear seats. The device includes: an acquisition module, a conversion module, a cropping module and an adjustment module.

[0017] The acquisition module is used to respond to the rear passenger's field of view display instruction to acquire the first seat information of the rear seat, the second seat information of the front seat and the first face information of the rear passenger. The rear seat is the seat where the rear passenger sits. The front seat is the seat that blocks the rear passenger's field of view.

[0018] The conversion module is used to convert the video in front of the vehicle collected by the camera deployed in front of the vehicle into a real-time image frame.

[0019] The cropping module is used to crop the picture information of the front seat blocking the vision of the rear passengers from the real-time image frame by using the first seat information, the second seat information and the first face information. The picture information includes the first blocking picture and the reduction ratio corresponding to the first blocking picture.

[0020] The adjustment module is used to adjust the first obstruction picture according to the reduction ratio, and display the adjusted first obstruction picture to the rear passengers through the vehicle screen located on the front seats.

[0021] In a third aspect, an embodiment of the present application provides a vehicle-mounted screen, which has the function of implementing the field of view display method based on the vehicle-mounted screen of the first aspect or any possible implementation method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0022] In a fourth aspect, an embodiment of the present application provides a vehicle display system, the system comprising: a vehicle-mounted screen as shown in the third aspect and at least one camera.

[0023] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer can execute the field of view display method based on the vehicle-mounted screen of the above-mentioned first aspect or any possible implementation method of the first aspect.

[0024] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the field of view display method based on a vehicle-mounted screen according to the first aspect or any possible implementation method.

[0025] Among them, the technical effects brought about by any design method in the second to sixth aspects can refer to the technical effects brought about by different possible implementation methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A structural schematic diagram of a vehicle display system provided in an embodiment of the present application; Figure 2 A structural schematic diagram of a vehicle-mounted screen provided in an embodiment of the present application; Figure 3 A schematic flow chart of a method for displaying a visual field based on a vehicle-mounted screen provided in an embodiment of the present application; Figure 4 A specific example diagram of a method for displaying a visual field based on a vehicle-mounted screen provided in an embodiment of the present application; Figure 5 A structural schematic diagram of a visual field display device based on a vehicle-mounted screen provided in an embodiment of the present application; Figure 6 Another structural schematic diagram of a vehicle display system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0029] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0030] At present, in-car screens are a common configuration in modern cars, especially in high-end models and luxury cars. These screens are usually installed behind the headrests of the front seats or in the roof lining, and are mainly used to play videos, music and other multimedia content to provide entertainment experience for rear passengers.

[0031] Although in-car screens are excellent for entertainment, they usually do not have the function of displaying the road conditions around the vehicle. Rear passengers cannot fully observe the road conditions in front and behind the vehicle due to the obstruction of the front seats. This limited vision not only affects the passengers' riding experience, but also reduces the safety of riding to a certain extent.

[0032] In an emergency, rear passengers lack awareness of the vehicle's surroundings and are unable to respond in time, increasing potential risks.

[0033] At the same time, existing in-vehicle screens mainly focus on one-way entertainment content playback, lack interaction with the vehicle's driving status and surrounding environment, and fail to fully utilize the potential of in-vehicle screens.

[0034] Based on this, the embodiment of the present application provides a method for displaying a field of view based on a vehicle screen, which is applied to a vehicle display system, and the vehicle display system includes a vehicle screen arranged on the back of a front seat for viewing by a rear passenger sitting in a rear seat. The method for displaying a field of view based on a vehicle screen includes: in response to a field of view display instruction of a rear passenger, obtaining first seat information of the rear seat, second seat information of the front seat, and first facial information of the rear passenger. The rear seat is a seat for the rear passenger to sit on. The front seat is a seat that blocks the field of view of the rear passenger. The video in front of the vehicle collected by a camera deployed in front of the vehicle is converted into a real-time image frame. Using the first seat information, the second seat information, and the first facial information, the picture information of the front seat blocking the field of view of the rear passenger is cut out from the real-time image frame. The picture information includes a first blocking picture and a reduction ratio corresponding to the first blocking picture. The first blocking picture is adjusted according to the reduction ratio, and the adjusted first blocking picture is displayed to the rear passenger through the vehicle screen located on the front seat.

[0035] The method for displaying the visual field based on the vehicle screen provided in the embodiment of the present application can display the visual field blocked by the front seats, so that the rear passengers can obtain a more comprehensive visual field in front, thereby improving the riding experience, especially during long-distance travel, and reducing the discomfort of the rear passengers. At the same time, the rear passengers can view the road conditions ahead through the vehicle screen, which helps to respond in time in an emergency and increases the safety of riding.

[0036] The solution provided by the embodiment of the present application will be described in detail below with reference to specific drawings.

[0037] On the one hand, the solution shown in the embodiment of the present application can be executed by a vehicle display system, and the vehicle display system can be deployed in a vehicle. Figure 1 As shown, the vehicle display system 100 may include a first camera 101 , a second camera 102 , and a vehicle screen 103 .

[0038] The first camera 101 is deployed in front of the vehicle, for example, it can be deployed on the top or bottom of the front windshield of the vehicle, and is used to capture a complete video in front of the vehicle and transmit the video to the vehicle screen. The first camera 101 can have a night vision function, which can ensure clear video capture in low-light environments such as night or foggy days.

[0039] The second camera 102 is deployed inside the vehicle, for example, it can be deployed on the top of the front seat of the vehicle, the side of the front seat or the roof of the vehicle, which is not limited in this application. The second camera 102 can be used to obtain facial information of rear passengers and body shape information of front passengers, etc.

[0040] The vehicle screen 103 is deployed on the front seat of the vehicle, for example, it can be directly installed on the back of the front seat, or it can be installed on the back of the front seat by moving the mobile bracket. The present application does not limit the installation position of the vehicle screen 103. The vehicle screen 103 is used to determine the processed picture after receiving the video sent by the first camera 101, the facial information of the rear passengers sent by the second camera 102, and the body shape information of the front passengers, and display the picture to the rear passengers.

[0041] It should be noted that the front seat can be the seat in front of the rear passenger where the rear passenger sits, or can be the seat in front of the rear passenger's field of vision. For example, when the rear passenger faces right, the rear passenger can adjust the vehicle screen 103 to the right of the rear passenger by operating the movable bracket of the vehicle screen 103. At this time, the seat to the right of the rear passenger is the seat in front of the rear passenger, and the passenger sitting on the seat to the right of the rear passenger is the front passenger of the rear passenger.

[0042] On the one hand, the embodiment of the present application provides a vehicle-mounted screen, such as Figure 2 As shown, the vehicle-mounted screen 103 includes a processor 201 , a memory 202 , a communication module 203 , and a display module 204 .

[0043] The processor 201 may include one or more processing units, for example, the processor 201 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0044] The memory 202 is used to store instructions and data. In some embodiments, the memory 202 may be a cache memory. The memory 202 may store instructions or data that have just been used or are cyclically used by the processor 201. If the processor needs to use the instructions or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 201, and improves the efficiency of the system.

[0045] The communication module 203 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the vehicle screen 103. The communication module 203 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The communication module 203 can be used to provide support for wireless communication between the vehicle screen 103 and the first camera 101 or between the vehicle screen 103 and the second camera 102.

[0046] The display module 204 can be used to provide the first blocked image generated after being processed by the processor 201 to the rear passengers.

[0047] It should be noted that the above Figure 1 The illustrated vehicle display system 100 and Figure 2 The illustrated vehicle-mounted screen 103 is only an example of the application scenario of the present application solution, and is not a limitation of the application scenario of the present application solution.

[0048] On the one hand, the embodiment of the present application provides a method for displaying a visual field based on a vehicle-mounted screen, which can be implemented by deploying Figure 2 The vehicle-mounted screen 103 shown in FIG. Figure 1 The vehicle display system 100 shown in FIG. Figure 3 As shown, the method includes: S301, in response to a rear passenger's field of view display instruction, obtaining first seat information of the rear seat, second seat information of the front seat, and first face information of the rear passenger.

[0049] Among them, the rear seats are seats for rear passengers, and the front seats are seats that block the vision of rear passengers.

[0050] It should be noted that the front seat can be the seat in front of the rear passenger or the seat in front of the rear passenger's field of vision. For example, when the rear passenger faces right, the rear passenger can adjust the vehicle screen to the right of the rear passenger by operating the mobile bracket of the vehicle screen. At this time, the seat to the right of the rear passenger is the seat in front of the rear passenger, and the passenger sitting on the seat to the right of the rear passenger is the front passenger of the rear passenger.

[0051] The first seat information includes first size information and first state information. The first state information includes a first distance between the front seat and the front windshield of the vehicle and a first tilt angle of the front seat, and the first tilt angle may be the first tilt angle adjusted by the front passenger automatically acquired. The second state information may include a second distance between the rear seat and the front windshield and a second tilt angle of the rear seat, and the second tilt angle may be the angle adjusted by the rear passenger automatically acquired.

[0052] A possible implementation method is to generate a field of view display instruction when it is detected that the rear passenger's line of sight is looking at the vehicle-mounted screen.

[0053] Another possible implementation is to generate a field of view display instruction when it is detected that a rear passenger touches the vehicle screen.

[0054] Another possible implementation method is to generate a field of view display instruction when it is detected that the rear passenger voice wakes up the in-vehicle screen.

[0055] In a possible implementation, the first size information and the second size information stored in the control system are read from the control system of the vehicle, the first state information and the second state information are acquired by using the vehicle screen, and the first face information is acquired by using the vehicle screen.

[0056] The first facial information includes the eye position of the rear passenger and the eye gaze position of the rear passenger.

[0057] S302, converting the video in front of the vehicle collected by a camera deployed in front of the vehicle into a real-time image frame.

[0058] One possible implementation method is that in response to the field of view display instruction of the rear passenger, a first camera deployed in the front of the vehicle collects the video in front of the vehicle and sends the video in front of the vehicle to the on-board screen. After receiving the video in front of the vehicle, the on-board screen extracts each frame of the image from the video in front of the vehicle to obtain the real-time image frame.

[0059] S303: Using the first seat information, the second seat information and the first face information, cut out the image information of the front seat blocking the field of vision of the rear passengers from the real-time image frame.

[0060] The picture information includes the first blocked picture and the reduction ratio corresponding to the first blocked picture.

[0061] In one possible implementation, the first distance, the first tilt angle, the second distance, the second tilt angle and the eye position are used to determine the sight line direction of the rear passenger by combining the three-dimensional geometric principle. The viewing picture of the rear passenger is determined by using the first distance, the first tilt angle, the second distance, the second tilt angle and the sight line direction.

[0062] The viewing image is the image viewed by the rear passengers in the direction of their sight that is not blocked by the front seats.

[0063] Specifically, a virtual camera model can be used to simulate the field of view of the rear passengers based on their line of sight, and then combined with the first distance, the first tilt angle, the second distance, and the second tilt angle, it is possible to determine the images that the rear passengers can view in the line of sight that are not blocked by the front seats.

[0064] The first blocking picture is cut out from the real-time image frame using the viewing picture, and the reduction ratio is determined according to the viewing picture, the real-time image frame and the first blocking picture.

[0065] For example, in the real-time image frame, the viewing image is used to mark the first blocked image blocked by the front seats, and then the real-time image frame is cropped to obtain the first blocked image. In order to accommodate the rear passengers, the viewing image is compared with the real-time image frame, and the reduction ratio of the first blocked image to be reduced is calculated. The reduction ratio can ensure that the rear passengers can clearly see the required image even if they are blocked by the front seats.

[0066] This process obtains the facial information of the rear passengers and can more accurately crop and adjust the picture to ensure that the displayed content is exactly what the rear passengers need to see, providing a personalized display experience.

[0067] Another possible implementation is that the front passenger is too tall and will exceed the front seat when sitting on the front seat. The second seat information may also include: the body size information of the front passenger. The first blocked image includes the second blocked image and the third blocked image. The second blocked image is an image in which the rear passenger's field of view is blocked by the front seat, and the third blocked image is an image in which the rear passenger is blocked by the front passenger.

[0068] In a possible implementation, using the viewing picture to cut out the first occlusion picture from the real-time image frame includes: using the viewing picture to cut out the second occlusion picture and the third occlusion picture from the real-time image frame; and combining the second occlusion picture and the third occlusion picture into the first occlusion picture.

[0069] S304: Adjust the first obstruction picture according to the reduction ratio, and display the adjusted first obstruction picture to the rear passengers through the vehicle screen located at the front seats.

[0070] In one possible implementation, historical eye gaze positions corresponding to multiple historical image frames before the real-time image frame within a preset time period are obtained. If the number of historical image frames corresponding to the same historical eye gaze position reaches a threshold, the focal length of the camera deployed in front of the vehicle is adjusted so that the target of the first occluded image corresponding to the gaze position is the highlighted target.

[0071] Exemplarily, when it is detected that the number of historical image frames corresponding to the historical eye gaze positions of the rear passenger reaches a quantity threshold within a preset time period, that is, the rear passenger gazes at the same position for a long time and the time reaches a time threshold, it can be determined that the rear passenger is interested in the target at this time, and the focus of the camera in front of the vehicle can be adjusted to display the target to the user as a highlighted target.

[0072] In a possible implementation, when the highlighted target is a building, detailed information of the building is obtained and displayed on the first obstructed image.

[0073] For example, Figure 4 As shown, when the highlighted target is a museum, detailed information such as the name, address, introduction, opening hours and closing hours of the museum can be displayed on the first shielding screen. For example, detailed information of the building can be displayed on the first shielding screen in the form of a list.

[0074] In a possible implementation, in response to a virtual display task submitted by a user on the vehicle screen, a virtual target is displayed on the first occluded screen, feedback information of the virtual target is determined based on the micro-expression information and detailed information, and the virtual display task is updated based on the feedback information.

[0075] For example, Figure 4As shown, the user can submit the virtual display task by clicking or voice wake-up on the in-vehicle screen, and display the virtual display target in the first blocked screen. The image of the virtual display target can be customized for the rear passengers. The feedback information of the virtual target can be determined based on the micro-expression information of the rear passengers and the detailed information of the building. For example, when the micro-expression information is used to determine that the rear passengers are interested in the museum, the detailed information of the museum can be displayed to the user by changing the action of the virtual target. Alternatively, when the micro-expression information is used to determine that the rear passengers are motion sickness, the display of the in-vehicle screen can also be processed to generate an anti-motion sickness mode to prevent the user from motion sickness.

[0076] A possible implementation method is to determine the reference object in front of the vehicle and the reference object position set in each historical image frame based on multiple historical image frames before the real-time image frame within a preset time period. According to the reference object position set, the relative movement trend between the reference object and the vehicle is determined. The relative movement trend includes relative position and relative speed. According to the relative position and relative speed, the real-time position of the reference object in the real-time image frame is predicted. If the distance difference between the real-time position and the current position of the vehicle reaches a distance threshold, a warning message is generated. The warning message is used to remind rear passengers that the vehicle is in danger.

[0077] Exemplarily, image data in front of the vehicle can be obtained from multiple historical image frames within a preset time period. Use object detection algorithms (such as YOLO, Faster R-CNN, etc.) to identify reference objects (such as pedestrians, other vehicles, obstacles, etc.) from historical image frames. Record the position of the reference object in each historical image frame to form a reference object position set. Based on the reference object position set, calculate the relative position and relative speed of the reference object and the vehicle. Based on the relative position and relative speed, predict the real-time position of the reference object in the real-time image frame. If the distance difference between the predicted real-time position and the current position of the vehicle reaches the distance threshold, generate a warning message to remind the rear passengers that the vehicle is in danger.

[0078] During this process, rear passengers can see the road conditions ahead, which helps them to respond in time in emergency situations and increase the safety of the ride. For example, when the vehicle brakes suddenly, rear passengers can prepare in advance. Warning information can enhance passengers' confidence in the safety of the vehicle and reduce anxiety and uneasiness caused by unknown risks. Rear passengers can understand the potential dangers around the vehicle and feel more at ease, especially on long journeys or complex road conditions. In emergency situations, such as impending collisions or emergency braking, warning information can remind passengers to prepare, such as fastening seat belts, staying seated, etc., to reduce the risk of injury. For the elderly and children, warning information can provide additional safety protection, reminding them to stay alert at critical moments. For passengers with limited mobility or special needs, warning information can provide necessary assistance to ensure their safety and comfort.

[0079] In a possible implementation, when the movement of the rear passenger is detected, the second facial information of the rear passenger is obtained, the image information is corrected using the second facial information, and the corrected image information is displayed to the rear passenger through the vehicle screen.

[0080] This process can be applied to the movement of rear passengers and has strong adaptability. When the rear passengers move, the second facial information of the rear passengers can be detected, and then the picture information can be cropped and adjusted to ensure that the rear passengers have the best field of view.

[0081] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the working principle of the vehicle-mounted screen. It can be understood that in order to realize the above functions, the vehicle-mounted screen includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0082] The embodiment of the present application can divide the vehicle screen into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processor. The above integrated module can be implemented in the form of hardware or software functional modules.

[0083] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. Figure 5 A possible schematic diagram of the composition of the visual field display device based on the vehicle-mounted screen involved in the above and embodiments is shown. Figure 5 As shown, the vehicle-mounted screen-based field of view display device 500 may include: an acquisition module 501 , a conversion module 502 , a cropping module 503 and an adjustment module 504 .

[0084] The acquisition module 501 is used to support the vehicle-mounted screen-based visual field display device 500 provided in the embodiment of the present application to perform the following Figure 3 S301 in the vehicle-mounted screen-based visual field display method shown.

[0085] The conversion module 502 is used to support the vehicle-mounted screen-based visual field display device 500 provided in the embodiment of the present application to perform the following Figure 3 S302 in the vehicle-mounted screen-based field of view display method is shown.

[0086] The cropping module 503 is used to support the vehicle-mounted screen-based visual field display device 500 provided in the embodiment of the present application to perform the following operations: Figure 3 S303 in the vehicle-mounted screen-based visual field display method shown.

[0087] The adjustment module 504 is used to support the vehicle-mounted screen-based visual field display device 500 provided in the embodiment of the present application to perform the following Figure 3 S304 in the vehicle-mounted screen-based visual field display method shown.

[0088] In a possible implementation, the first seat information includes first size information and first state information. The second seat information includes second size information and second state information. The acquisition module is specifically used to read the first size information and second size information stored in the control system from the control system of the vehicle. The first state information and the second state information are acquired using the vehicle screen. The first state information includes a first distance from the front windshield of the vehicle and a first tilt angle. The second state information includes a second distance from the front windshield and a second tilt angle. The first facial information is acquired using the vehicle screen.

[0089] In a possible implementation, the first facial information includes the eye position. The cropping module is specifically used to determine the sight line direction of the rear passenger using the first distance, the first tilt angle, the second distance, the second tilt angle and the eye position. The viewing picture of the rear passenger is determined using the first distance, the first tilt angle, the second distance, the second tilt angle and the sight line direction. The viewing picture is the picture that the rear passenger sees in the sight line direction and is not blocked by the front row seats. The viewing picture is used to crop the first blocked picture from the real-time image frame, and the reduction ratio is determined according to the viewing picture, the real-time image frame and the first blocked picture.

[0090] In a possible implementation, the second seat information also includes: body size information of the front passenger, the front passenger is the passenger sitting in the front seat. The first obstruction picture includes the second obstruction picture and the third obstruction picture, the second obstruction picture is a picture in which the field of view of the rear passenger is blocked by the front seat, and the third obstruction picture is a picture in which the rear passenger is blocked by the front passenger. The cropping module is specifically used to crop the second obstruction picture and the third obstruction picture from the real-time image frame using the viewing picture. The second obstruction picture and the third obstruction picture are combined into the first obstruction picture.

[0091] In a possible implementation, the first facial information also includes an eye gaze position. The first obstructed image includes a highlighted target. The device can also be used to obtain historical eye gaze positions corresponding to multiple historical image frames before the real-time image frame within a preset time period. If the number of historical image frames corresponding to the same historical eye gaze position reaches a number threshold, the focal length of the camera deployed in front of the vehicle is adjusted so that the target of the first obstructed image corresponding to the gaze position is the highlighted target.

[0092] In a possible implementation, the device can also be used to obtain detailed information of a building when the highlighted target is a building, and display the detailed information on the first shielding picture.

[0093] In a possible implementation, the first facial information includes micro-expression information. The device can also be used to display a virtual target on the first occluded screen in response to a virtual display task submitted by a user on the vehicle screen. According to the micro-expression information and detailed information, feedback information of the virtual target is determined, and the virtual display task is updated according to the feedback information.

[0094] In a possible implementation, the device can also be used to determine the reference object in front of the vehicle and the reference object position set in each historical image frame based on multiple historical image frames before the real-time image frame within a preset time period. According to the reference object position set, the relative movement trend between the reference object and the vehicle is determined. The relative movement trend includes relative position and relative speed. According to the relative position and relative speed, the real-time position of the reference object in the real-time image frame is predicted.

[0095] If the distance difference between the real-time position and the current position of the vehicle reaches the distance threshold, a warning message is generated. The warning message is used to remind the rear passengers that the vehicle is in danger.

[0096] In a possible implementation, the device can also be used to obtain the second facial information of the rear passenger when the rear passenger moves, use the second facial information to correct the image information, and display the corrected image information to the rear passenger through the vehicle screen.

[0097] In a possible implementation, the device can also be used to generate the field of view display instruction when it is detected that the rear passenger's line of sight is looking at the vehicle-mounted screen.

[0098] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0099] The visual field display device 500 based on the vehicle-mounted screen provided in the embodiment of the present application is used to perform the above Figure 3 The visual field display method based on the vehicle-mounted screen shown can therefore achieve the same effect as the visual field display method based on the vehicle-mounted screen described above.

[0100] The embodiment of the present application also provides a vehicle-mounted screen, which can be the vehicle-mounted screen described in the above embodiment. Figure 2 The vehicle-mounted screen 103 shown can execute the field of view display method based on the vehicle-mounted screen and related steps in the above method embodiment.

[0101] An embodiment of the present application also provides a computer-readable storage medium having instructions stored thereon, which, when executed, execute the field of view display method based on the vehicle-mounted screen and related steps in the above-mentioned method embodiment.

[0102] An embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer executes the field of view display method based on the vehicle-mounted screen and related steps in the above method embodiment.

[0103] In some embodiments, the methods described herein may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles of manufacture.

[0104] The present application also provides a vehicle display system 600, such as Figure 6 As shown, the vehicle display system 600 includes at least one processor 601 and at least one interface circuit 602 .

[0105] As an example, when the vehicle display system 600 includes a processor and an interface circuit, the processor may be Figure 6 The processor 601 shown in the solid line frame (or the processor 601 shown in the dotted line frame) may be Figure 6 The interface circuit 602 shown in the solid line frame (or the interface circuit 602 shown in the dashed line frame) is shown in the solid line frame. When the vehicle display system 600 includes two processors and two interface circuits, the two processors include Figure 6 The processor 601 shown in the solid line frame and the processor 601 shown in the dotted line frame, the two interface circuits include Figure 6 The interface circuit 602 shown in the solid line frame and the interface circuit 602 shown in the dotted line frame are not limited to this.

[0106] The processor 601 and the interface circuit 602 can be interconnected via a line. For example, the interface circuit 602 can be used to receive a signal. For another example, the interface circuit 602 can be used to send a signal to another device (such as the processor 601). For example, the interface circuit 602 can read a computer instruction stored in a memory and send the computer instruction to the processor 601. The processor 601 executes the instruction and implements the various steps in the above embodiments in combination with the input and output devices, such as implementing Figure 3 or Figure 4 The various steps performed in the method embodiment shown. Of course, the vehicle display system 600 may also include other discrete components, which are not specifically limited in the embodiment of the present application.

[0107] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0108] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a vehicle screen (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0112] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for displaying visual field based on a vehicle-mounted screen, characterized in that: Applied to a vehicle display system, the vehicle display system includes a vehicle screen arranged on the back of a front seat for rear passengers to watch, the method includes: In response to a rear passenger's field of view display instruction, first seat information of a rear seat, second seat information of a front seat, and first face information of a rear passenger are acquired; the rear seat is a seat on which the rear passenger sits; and the front seat is a seat that blocks the rear passenger's field of view; Converting a video in front of the vehicle captured by a camera deployed in front of the vehicle into a real-time image frame; Using the first seat information, the second seat information and the first face information, cropping the picture information of the front seat blocking the field of vision of the rear passenger from the real-time image frame; the picture information includes the first blocking picture and the reduction ratio corresponding to the first blocking picture; The first shielding picture is adjusted according to the reduction ratio, and the adjusted first shielding picture is displayed to the rear passengers through the vehicle screen located on the front seats.

2. The method according to claim 1, characterized in that The first seat information includes first size information and first state information; the second seat information includes second size information and second state information; The obtaining of first seat information of a rear seat, second seat information of a front seat, and first facial information of a rear passenger includes: Reading the first size information and the second size information stored by the control system from the control system of the vehicle; The first state information and the second state information are acquired by using the vehicle-mounted screen; the first state information includes a first distance from the front windshield of the vehicle and a first tilt angle; the second state information includes a second distance from the front windshield and a second tilt angle; The first facial information is acquired using the vehicle-mounted screen.

3. The method according to claim 1, characterized in that The first facial information includes eye positions; the first seat information, the second seat information, and the first facial information are used to crop a first blocking picture in which the front seat blocks the field of vision of the rear passenger from the real-time image frame, including: Determining the sight direction of the rear passenger using the first distance, the first tilt angle, the second distance, the second tilt angle and the eye position; Determine the viewing picture of the rear-row passenger by using the first distance, the first tilt angle, the second distance, the second tilt angle and the sight line direction; the viewing picture is the picture viewed by the rear-row passenger in the sight line direction and not blocked by the front-row seat; The first blocking picture is cropped from the real-time image frame using the viewing picture, and the reduction ratio is determined according to the viewing picture, the real-time image frame and the first blocking picture.

4. The method according to claim 3, characterized in that The second seat information further includes: body size information of a front-row passenger, wherein the front-row passenger is a passenger sitting on the front-row seat; the first blocked picture includes a second blocked picture and a third blocked picture, wherein the second blocked picture is a picture in which the field of view of the rear-row passenger is blocked by the front-row seat, and the third blocked picture is a picture in which the rear-row passenger is blocked by the front-row passenger; The step of cutting out the first blocked picture from the real-time image frame using the viewing picture includes: Using the viewing picture, cropping the second blocked picture and the third blocked picture from the real-time image frame; The second occlusion picture and the third occlusion picture are combined into the first occlusion picture.

5. The method according to claim 1, characterized in that The first facial information further includes an eye gaze position; the first occluded image includes a highlighted target; and the method further includes: Obtaining historical eye gaze positions corresponding to a plurality of historical image frames preceding the real-time image frame within a preset time period; If the number of historical image frames corresponding to the same historical eye gaze position reaches a quantity threshold, the focal length of the camera deployed in front of the vehicle is adjusted so that the target of the first blocked image corresponding to the gaze position is the highlighted display target.

6. The method according to claim 5, characterized in that The method further comprises: When the highlighted target is a building, obtaining detailed information of the building; The detailed information is displayed on the first shielding screen.

7. The method according to claim 6, characterized in that The first facial information includes micro-expression information; the method further includes: In response to a virtual display task submitted by a user on the in-vehicle screen, displaying a virtual target on the first obstruction picture; Determine feedback information of the virtual target according to the micro-expression information and the detailed information, and update the virtual display task according to the feedback information.

8. The method according to claim 1, characterized in that: The method further comprises: Determine, according to a plurality of historical image frames preceding the real-time image frame within a preset time period, a reference object in front of the vehicle and a reference object position set of the reference object in each of the historical image frames; Determining the relative motion trend between the reference object and the vehicle according to the reference object position set; the relative motion trend includes relative position and relative speed; Predicting the real-time position of the reference object in the real-time image frame according to the relative position and the relative speed; If the distance difference between the real-time position and the current position of the vehicle reaches a distance threshold, a warning message is generated; the warning message is used to remind the rear passengers that the vehicle is in danger.

9. The method according to claim 1, characterized in that: The method further comprises: When movement of the rear-seat passenger is detected, acquiring second facial information of the rear-seat passenger; The picture information is corrected using the second facial information, and the corrected picture information is displayed to the rear passengers via the vehicle-mounted screen.

10. The method according to claim 1, characterized in that The method further comprises: When it is detected that the rear passenger's line of sight is directed toward the vehicle-mounted screen, the field of view display instruction is generated.

11. A visual field display device based on a vehicle-mounted screen, characterized in that: Applied to a vehicle display system, the vehicle display system includes a vehicle screen arranged on the back of the front seat for rear passengers to watch, the device includes: an acquisition module, for acquiring, in response to a rear passenger's field of view display instruction, first seat information of a rear seat, second seat information of a front seat, and first face information of a rear passenger; the rear seat is a seat on which the rear passenger sits; the front seat is a seat that blocks the rear passenger's field of view; A conversion module, used to convert the video in front of the vehicle collected by a camera deployed in front of the vehicle into a real-time image frame; A cropping module, used to crop the picture information of the front seat blocking the field of vision of the rear passenger from the real-time image frame by using the first seat information, the second seat information and the first face information; the picture information includes a first blocked picture and a reduction ratio corresponding to the first blocked picture; The adjustment module is used to adjust the first shielding picture according to the reduction ratio, and display the adjusted first shielding picture to the rear passengers through the vehicle screen located on the front seats.

12. A vehicle-mounted screen, characterized in that: The vehicle-mounted screen includes a processor and a memory, the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the field of view display method based on the vehicle-mounted screen according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to implement the field of view display method based on the vehicle-mounted screen as described in any one of claims 1-10.

Images