Method, device and storage medium for blind area monitoring

Abstract

This application discloses a method, apparatus, and storage medium for blind spot monitoring, belonging to the field of vehicle control technology. The method includes: acquiring multiple corrected images of the vehicle's surroundings; converting the viewing angle of the multiple corrected images to a simulated observation viewpoint to obtain multiple virtual images of the vehicle under the simulated observation viewpoint; stitching together each virtual image to obtain a complete image of the vehicle's surroundings; and displaying the blind spot portion of the complete image of the vehicle's surroundings in response to a trigger condition that the vehicle's operating conditions meet for blind spot image display. This enables the real-time display of a complete blind spot image using existing vehicle equipment when the driver requires blind spot monitoring, thereby reducing safety hazards caused by blind spots during vehicle operation without increasing manufacturing costs or after-sales maintenance difficulties.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method, device and storage medium for blind spot monitoring. Background Technology

[0002] As car ownership continues to increase, users' demands for vehicle intelligence and safety are also gradually rising. Blind spot monitoring, as an important part of active vehicle safety technology, can reduce safety hazards during vehicle operation.

[0003] In related technologies, microwave radar or side-view cameras are used to monitor blind spots. Microwave radar relies solely on obstacles to trigger audible alerts, preventing drivers from directly seeing the specific conditions of the blind spot and thus hindering appropriate vehicle adjustments. Side-view cameras are expensive and complex to install, increasing manufacturing costs and after-sales maintenance difficulties. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for blind spot monitoring, which can be used to reduce safety hazards caused by blind spots during vehicle operation without increasing manufacturing costs or after-sales maintenance difficulties. The technical solution is as follows: On one hand, embodiments of this application provide a method for blind spot monitoring, the method comprising: Acquire multi-channel corrected images of the area surrounding the vehicle; The perspective of the multi-channel corrected images is converted to the simulated observation perspective to obtain multiple virtual images of the vehicle under the simulated observation perspective; Each virtual image is stitched together to obtain a complete image of the area around the vehicle; In response to the vehicle operating conditions meeting the triggering conditions for blind spot image display, the blind spot portion of the complete image around the vehicle is displayed.

[0005] On the other hand, a device for blind spot monitoring is provided, the device comprising: The first acquisition module is used to acquire multi-channel corrected images around the vehicle; The conversion module is used to convert the perspective of the multi-channel corrected images to the simulated observation perspective, so as to obtain multiple virtual images of the vehicle under the simulated observation perspective; The stitching module is used to stitch together each virtual image to obtain a complete image of the area around the vehicle. The display module is used to display the blind spot portion of the complete image around the vehicle in response to the triggering condition that the vehicle's operating conditions meet the blind spot image display requirement.

[0006] On the other hand, a non-transitory computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement any of the above-described blind zone monitoring methods.

[0007] On the other hand, a computer program product is also provided, the computer program product including computer instructions, which, when executed by a processor, implement the steps of any of the above-described blind zone monitoring methods.

[0008] On the other hand, a vehicle is also provided, the vehicle comprising: a vehicle body; and a cockpit domain controller, the cockpit domain controller being configured to perform the steps of the blind spot monitoring method described above.

[0009] The technical solution provided in this application brings at least the following beneficial effects: This application acquires multiple corrected images of the vehicle's surroundings and converts them into multiple virtual images of the vehicle from a simulated observation perspective. This facilitates subsequent stitching of images from different directions within the same viewpoint. Each virtual image is then stitched together to obtain a complete image of the vehicle's surroundings. Furthermore, by monitoring the vehicle's operating conditions, it determines whether the conditions meet the triggering conditions for blind spot image display. When these conditions are met, the blind spot portion of the complete image surrounding the vehicle is displayed, ensuring that the blind spot image is only triggered when needed, thus avoiding driver distraction. This allows for the real-time display of a complete blind spot image using existing vehicle equipment when the driver requires blind spot monitoring. This reduces blind spot safety hazards during vehicle operation and improves overall driving safety without increasing manufacturing costs or after-sales maintenance complexity. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart of a blind spot monitoring method provided in an embodiment of this application; Figure 3 This is a logic flowchart of a blind spot monitoring method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a blind spot monitoring device provided in an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0013] This application provides a method for blind spot monitoring. Please refer to... Figure 1 The diagram illustrates the implementation environment of the method provided in this application embodiment. This implementation environment may include: a cockpit domain controller 11, a microwave radar 12, a body controller 13, an in-vehicle surround-view camera group 14, a center console display screen 15, an instrument panel display screen 16, and a gateway 17.

[0014] Optionally, the microwave radar 12 is installed outside the vehicle at a location that ensures microwave coverage of the vehicle's blind spot, and is used to acquire obstacle detection results in the vehicle's blind spot and send them to the cockpit domain controller 11. The obstacle detection results in the vehicle's blind spot are used to indicate whether there are obstacles in the vehicle's blind spot and the distance between the obstacles and the vehicle. The body controller 13 is used to acquire the on / off status of the turn signals and send them to the cockpit domain controller 11. The on / off status of the turn signals includes an on state or a off state.

[0015] In one possible implementation, the vehicle surround-view camera group 14 includes a left surround-view camera, a right surround-view camera, and a rear surround-view camera; the vehicle surround-view camera group 14 can be a wide-angle fisheye camera with high dynamic range and low-light enhancement functions; the left surround-view camera can be installed below the left rearview mirror of the vehicle, the right surround-view camera can be installed below the right rearview mirror of the vehicle, and the rear surround-view camera can be installed at the rear of the vehicle; the vehicle surround-view camera group 14 is used to acquire multiple raw images around the vehicle and send them to the cockpit domain controller 11.

[0016] For example, the display screen 15 on the center console or the display screen 16 on the instrument panel is used to display the blind spot portion of the complete image around the vehicle. The display interface can be in the form of a floating window, which is above the original display interface of the screen and can be dragged at will. The display interface also includes distance reference lines between obstacles and the vehicle and warning icons.

[0017] Optionally, the cockpit domain controller 11, microwave radar 12, body controller 13, vehicle surround view camera group 14, center console display screen 15, and instrument panel display screen 16 establish communication connections via wired or wireless networks. For example, the vehicle surround view camera group 14 and the cockpit domain controller 11 can communicate via video cable; the cockpit domain controller 11 can communicate with the center console display screen 15 and the instrument panel display screen 16 via video cable; and the microwave radar 12 communicates with the cockpit domain controller 11 via a CAN bus through gateway 17.

[0018] Based on the above Figure 1 The implementation environment shown in this application provides a method for blind spot monitoring. Figure 2 As shown, taking the application of this method to the cockpit domain controller as an example, the method includes steps 201-204.

[0019] In step 201, the cockpit domain controller acquires multi-channel corrected images of the vehicle's surroundings.

[0020] In one possible implementation, the cockpit domain controller acquires multiple corrected images of the vehicle's surroundings, including: acquiring multiple original images of the vehicle's surroundings; and performing distortion correction processing on the multiple original images to obtain multiple corrected images.

[0021] For example, the cockpit domain controller acquires multiple raw images of the vehicle's surroundings via an onboard surround-view camera array. In one possible implementation, the onboard surround-view camera array includes a left-side surround-view camera, a right-side surround-view camera, and a rear-side surround-view camera; the left-side, right-side, and rear-side surround-view cameras can be wide-angle fisheye cameras with high dynamic range and low-light enhancement capabilities; the left-side surround-view camera can be mounted below the vehicle's left-side rearview mirror, the right-side surround-view camera can be mounted below the vehicle's right-side rearview mirror, and the rear-side surround-view camera can be mounted at the rear of the vehicle.

[0022] Optionally, after acquiring multiple raw images around the vehicle, the cockpit domain controller performs distortion correction processing on the multiple raw images to obtain multiple corrected images. This includes: the cockpit domain controller reverse-maps the distorted multiple raw images corresponding to each vehicle surround view camera to the distortion-free multiple corrected images corresponding to that vehicle surround view camera, based on the information from each vehicle surround view camera. The multiple corrected images include the corrected image on the left side of the vehicle, the corrected image on the right side of the vehicle, and the corrected image on the rear side of the vehicle.

[0023] In one possible implementation, the cockpit domain controller determines the information of the vehicle surround-view camera by reading pre-stored information, wherein the information of the vehicle surround-view camera includes a second internal parameter and a second external parameter; then, it determines a mapping table for distortion correction based on the second internal parameter in the information of the vehicle surround-view camera; and based on the distorted multiple original images, combined with the mapping table for distortion correction, it obtains distort-free multiple corrected images. Optionally, the correspondence between the camera's internal parameters and the mapping table for distortion correction can be preset based on experience.

[0024] In step 202, the cockpit domain controller converts the perspective of the multi-channel corrected images to the simulated observation perspective, thereby obtaining multiple virtual images of the vehicle under the simulated observation perspective.

[0025] For example, after acquiring the multi-channel corrected images around the vehicle, before switching the viewpoint of the multi-channel corrected images around the vehicle to the simulated observation viewpoint, information of the virtual camera and the vehicle-mounted surround view camera corresponding to the simulated observation viewpoint are acquired. The information of the virtual camera includes a first internal parameter and a first external parameter, and the information of the vehicle-mounted surround view camera includes a second internal parameter and a second external parameter. The homography matrix corresponding to the virtual camera and the vehicle-mounted surround view camera is calculated based on the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter. The homography matrix is ​​used to switch the viewpoint from the multi-channel corrected images to the simulated observation viewpoint through the viewpoint conversion between the cameras.

[0026] Optionally, the second internal parameters include the focal length, principal point coordinates, and tilt factor of the vehicle surround-view camera; the second external parameters include the position and orientation of the vehicle surround-view camera in the world coordinate system. The orientation of the vehicle surround-view camera in the world coordinate system can be represented by a second rotation matrix, and the position of the vehicle surround-view camera in the world coordinate system can be represented by the coordinates of the optical center of the vehicle surround-view camera in the world coordinate system. The center of the world coordinate system can be set empirically, for example, it can be set as the geometric center point of the vehicle.

[0027] Optionally, the information of the virtual camera can be set based on experience. The information of the virtual camera includes a first internal parameter and a first external parameter. The first internal parameter includes the focal length, principal point coordinates, and tilt factor of the virtual camera. The first external parameter includes the position and orientation of the virtual camera in the world coordinate system. The orientation of the virtual camera in the world coordinate system can be represented by a first rotation matrix, and the position of the virtual camera in the world coordinate system can be represented by the coordinates of the optical center of the virtual camera in the world coordinate system.

[0028] In one possible implementation, the virtual camera's position in the world coordinate system can be the coordinates corresponding to a preset position below the vehicle's right rearview mirror; the virtual camera's orientation in the world coordinate system can be a preset angle between the vehicle's side and rear and the ground; the virtual camera's principal point coordinates and tilt factor can be set according to commonly used engineering ideal values; the virtual camera's focal length can be preset based on the driver's requirements for blind spot range. For example, the preset position and preset angle can be set based on experience; the commonly used engineering ideal values ​​and the driver's requirements for blind spot range can be predetermined through cloud-based big data.

[0029] Optionally, after determining the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter, the homography matrix corresponding to the virtual camera and the vehicle surround view camera is calculated based on the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter. This includes substituting the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter into a first calculation formula or a second calculation formula for calculating the homography matrix to obtain the homography matrix corresponding to the virtual camera and the vehicle surround view camera. The first calculation formula and the second calculation formula are shown below:

[0030] It is a homography matrix; This is the first intrinsic parameter matrix; This is the first rotation matrix; This is the second intrinsic parameter matrix. Used to map pixel coordinates in a multi-path corrected image to a direction vector in a normalized camera coordinate system; This is the second rotation matrix. Used to rotate the direction vector in the normalized camera coordinate system back to the world coordinate system.

[0031]

[0032] and for The first two columns; Let the coordinates of the optical center of the virtual camera in the world coordinate system be denoted as . The coordinates of the optical center of the vehicle-mounted surround-view camera in the world coordinate system.

[0033] For example, after calculating the homography matrix corresponding to the virtual camera and the vehicle surround view camera, the viewpoint of the multi-channel corrected image is switched to the simulated observation viewpoint to obtain multiple virtual images of the vehicle under the simulated observation viewpoint. This includes: performing viewpoint conversion on each pixel in the multi-channel corrected image through the homography matrix to obtain multiple virtual images of the vehicle under the simulated observation viewpoint.

[0034] In one possible implementation, the cockpit domain controller calculates the coordinates of each pixel in any corrected image around the vehicle in the world coordinate system, multiplies it by the result of the homography matrix calculation, and uses the result as the virtual image of the vehicle corresponding to that corrected image under the simulated observation view. The same method is used to process each corrected image to obtain multiple virtual images of the vehicle under the simulated observation view.

[0035] In step 203, the cockpit domain controller stitches together each virtual image to obtain a complete image of the vehicle's surroundings.

[0036] Optionally, after acquiring multiple virtual images of the vehicle from a simulated observation perspective, before stitching each virtual image together, a weight distribution map corresponding to each virtual image is acquired. The weight distribution map is used to describe the position weight of each pixel in the virtual image. For the overlapping area between multiple virtual images, the closer a pixel is to the center of the virtual image, the higher its position weight.

[0037] For example, the cockpit domain controller obtains a weight distribution map corresponding to each virtual image, including: for any virtual image, generating a weight distribution map based on experience, with the rule that pixels closer to the center of the virtual image have higher positional weights. Specifically, for overlapping areas between multiple virtual images, pixels that belong to the same location in the actual scene should have a higher positional weight in one virtual image than in other virtual images, given they are closer to the center.

[0038] In one possible implementation, after obtaining the weight distribution map corresponding to each virtual image, the cockpit domain controller performs stitching processing on each virtual image to obtain a complete image of the vehicle's surroundings. This includes: for the diagonal region of the simulated observation viewpoint, increasing the position weight of pixels belonging to the diagonal region in the virtual image corresponding to the corrected image behind the vehicle; calculating the pixel value of the first pixel in the complete image of the vehicle's surroundings based on the pixel value and position weight of each first pixel in the overlapping area of ​​the image in different virtual images; using the pixel value of each second pixel in the non-overlapping area of ​​the image in the virtual image as the pixel value of the second pixel in the complete image of the vehicle's surroundings; and generating a complete image of the vehicle's surroundings based on the pixel values ​​of the first pixel and the second pixel in the complete image of the vehicle's surroundings.

[0039] Optionally, taking the virtual camera's position as below the right rearview mirror of the vehicle as an example, the simulated observation angle is the angle observed by the virtual camera from below the right rearview mirror. In this case, the diagonal area of ​​the simulated observation angle is the area of ​​the upper left corner of the vehicle.

[0040] For example, for the diagonal region of the simulated observation viewpoint, increasing the position weight of pixels belonging to the diagonal region in the virtual image corresponding to the corrected image of the rear of the vehicle includes: the cockpit domain controller increasing the position weight of pixels belonging to the upper left corner of the vehicle in the virtual image corresponding to the corrected image of the rear of the vehicle to a preset level based on the position weights contained in the weight distribution map corresponding to the virtual image. For example, the preset level can be 100%.

[0041] In one possible implementation, for overlapping areas, the pixel value of each first pixel in the complete image surrounding the vehicle is calculated based on its pixel value and position weight in different virtual images. This includes: the cockpit domain controller substituting the pixel value and position weight of each first pixel in different virtual images into a pixel value weighted fusion calculation formula to obtain the pixel value of the first pixel in the complete image surrounding the vehicle. The pixel value weighted fusion calculation formula is as follows:

[0042] The pixel value of the first pixel in the complete image around the vehicle; The position weight of the first pixel in the virtual image corresponding to the corrected image on the left side of the vehicle; The position weight of the first pixel in the virtual image corresponding to the corrected image on the right side of the vehicle; The position weight of the first pixel in the virtual image corresponding to the corrected image at the rear of the vehicle; The pixel value of the first pixel in the corrected image on the right side of the vehicle; The pixel value of the first pixel in the corrected image on the left side of the vehicle; This represents the pixel value of the first pixel in the corrected image at the rear of the vehicle. Optionally, if any first pixel does not exist in any virtual image, the corresponding position weight and pixel value are set to 0 during calculation.

[0043] For example, for non-overlapping areas of the image, the cockpit domain controller directly uses the pixel value of each second pixel in the virtual image as the pixel value of the second pixel in the complete image around the vehicle. After obtaining the pixel values ​​of the first pixel in the complete image around the vehicle and the pixel values ​​of the second pixel in the complete image around the vehicle, the cockpit domain controller stitches together the pixel values ​​of the first pixel in the complete image around the vehicle and the pixel values ​​of the second pixel in the complete image around the vehicle to generate the complete image around the vehicle.

[0044] In step 204, in response to the vehicle operating conditions meeting the triggering conditions for blind spot image display, the cockpit domain controller displays the blind spot portion in the complete image around the vehicle.

[0045] In one possible implementation, the method of determining whether the vehicle's operating conditions meet the trigger conditions for displaying a blind spot image includes: obtaining the risk level of the vehicle's blind spot and the on / off state of the turn signal, wherein the on / off state of the turn signal includes an on state or a off state; and in response to the risk level of the vehicle's blind spot being greater than a level threshold or the turn signal switching from an off state to an on state, indicating that the vehicle's operating conditions meet the trigger conditions for displaying a blind spot image.

[0046] For example, the cockpit domain controller obtains the risk level of the vehicle's blind spot by acquiring obstacle detection results of the vehicle's blind spot via microwave radar. These obstacle detection results indicate whether an obstacle exists in the vehicle's blind spot and the distance between the obstacle and the vehicle. In one possible implementation, if the detection result indicates that no obstacle exists in the vehicle's blind spot, the risk level of the blind spot is indicated as level zero; if the detection result indicates that an obstacle exists in the vehicle's blind spot and the distance between the obstacle and the vehicle is within a first threshold range, the risk level of the vehicle's blind spot is indicated as level one; if the detection result indicates that an obstacle exists in the vehicle's blind spot and the distance between the obstacle and the vehicle is within a second threshold range, the risk level of the vehicle's blind spot is indicated as level two.

[0047] Optionally, the microwave radar is installed outside the vehicle at a location that ensures the microwaves can cover the vehicle's blind spot. The vehicle's blind spot is the area to the side and rear of the vehicle that the driver cannot directly observe or view through the rearview mirror, and can be defined based on experience. The first threshold range and the second threshold range can be set based on experience, and the distance included in the first threshold range needs to be greater than the distance included in the second threshold range. In one possible implementation, the vehicle's blind spot can be a pre-marked area to the right rear of the vehicle; the first threshold range can be 5 to 8 meters; and the second threshold range can be 3 to 5 meters.

[0048] For example, the cockpit domain controller obtains the on / off state of the turn signal by means of: the cockpit domain controller can obtain the on / off state of the turn signal through the body controller, wherein the on / off state of the turn signal includes an on state or a off state.

[0049] In one possible implementation, after obtaining the risk level of the vehicle's blind spot and the on / off state of the turn signals, the risk level of the vehicle's blind spot is compared with a level threshold. If at least one of the following conditions is met: the risk level of the vehicle's blind spot is greater than the level threshold or the turn signal switches from off to on, the vehicle's operating condition is indicated to meet the trigger condition for displaying the blind spot image. If the risk level of the vehicle's blind spot is not greater than the level threshold or the turn signal does not switch from off to on, the vehicle's operating condition is indicated to not meet the trigger condition for displaying the blind spot image. The level threshold can be set empirically; for example, it can be set to level zero.

[0050] Optionally, if the vehicle's operating conditions meet the triggering conditions for blind spot image display, the blind spot portion in the complete image around the vehicle is determined by combining the stitched complete image and the pre-set blind spot area. The cockpit domain controller displays the blind spot portion in the complete image around the vehicle, including: the cockpit domain controller displays the blind spot portion in the complete image around the vehicle through the display screen of the vehicle's center console or the instrument panel.

[0051] Optionally, the display interface can take the form of a floating window, which sits above the original display screen and can be dragged freely. The display interface also includes distance reference lines between the obstacle and the vehicle, and warning icons. The warning icons are used to indicate when an obstacle enters a first threshold range and a second threshold range of the vehicle. The floating window can remain active for a preset duration after activation, and can be manually closed by the driver within that preset duration, or it can close when the obstacle disappears. In one possible implementation, the preset duration can be set empirically, for example, it can be set to any duration between 5 and 8 seconds.

[0052] For example, when the vehicle type changes, it is necessary to recalibrate the second internal and second external parameters of the onboard surround-view camera, as well as the first internal and first external parameters of the virtual camera. Optionally, the calibration process can employ a target calibration method or an automatic calibration method based on natural features. In one possible implementation, the cockpit domain controller can also store the second internal and second external parameters of the surround-view camera corresponding to different vehicle models through internally integrated non-volatile memory, facilitating rapid adaptation after switching vehicle models.

[0053] In summary, with Figure 3 The following is an example of the logic flowchart for blind spot monitoring. Optionally, the executing entity can be the cockpit domain controller. Step 301: Acquire original images of the vehicle's surroundings using the left-side surround-view camera, right-side surround-view camera, and rear-side surround-view camera; Step 302: Perform distortion correction processing on the original images based on the information from each vehicle-mounted surround-view camera to obtain multi-channel corrected images; Step 303: Perform virtual perspective conversion on the multi-channel corrected images to obtain multiple virtual images; Step 304: Stitch and optimize the virtual images to obtain a complete image of the vehicle's surroundings; Step 305: Display the blind spot portion in the complete image of the vehicle's surroundings.

[0054] This application's embodiments acquire multiple corrected images of the vehicle's surroundings and convert them into multiple virtual images of the vehicle from a simulated observation perspective. This facilitates subsequent stitching of images from different directions within the same viewpoint. Each virtual image is then stitched together to obtain a complete image of the vehicle's surroundings. Furthermore, by monitoring the vehicle's operating conditions, it is determined whether the conditions meet the triggering conditions for blind spot image display. When the conditions are met, the blind spot portion of the complete image surrounding the vehicle is displayed, ensuring that the blind spot image is only triggered when needed, thus avoiding driver distraction. This achieves real-time display of a complete blind spot image using existing vehicle equipment when the driver requires blind spot monitoring, thereby reducing blind spot safety hazards during vehicle operation and improving overall driving safety without increasing manufacturing costs or after-sales maintenance difficulties.

[0055] See Figure 4 This application provides a blind spot monitoring device, which includes: The first acquisition module 401 is used to acquire multi-channel corrected images around the vehicle; The conversion module 402 is used to convert the viewpoint of the multi-channel corrected images to the simulated observation viewpoint, so as to obtain multiple virtual images of the vehicle under the simulated observation viewpoint. The stitching module 403 is used to stitch together each virtual image to obtain a complete image of the area around the vehicle. Display module 404 is used to display the blind spot portion of the complete image around the vehicle in response to the triggering condition of the vehicle operating conditions meeting the blind spot image display.

[0056] In one possible implementation, the conversion module 402 is used to perform viewpoint conversion on each pixel in the multi-path corrected image using a homography matrix to obtain multiple virtual images of the vehicle under a simulated observation viewpoint.

[0057] In one possible implementation, the conversion module 402 is further configured to acquire information about the virtual camera and the vehicle-mounted surround-view camera corresponding to the simulated observation viewpoint. The information about the virtual camera includes a first internal parameter and a first external parameter, and the information about the vehicle-mounted surround-view camera includes a second internal parameter and a second external parameter. The homography matrix corresponding to the virtual camera and the vehicle-mounted surround-view camera is calculated based on the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter. The homography matrix is ​​used to convert the viewpoint from the multi-channel corrected image to the simulated observation viewpoint through the viewpoint conversion between the cameras.

[0058] In one possible implementation, the stitching module 403 is also used to obtain a weight distribution map corresponding to each virtual image. The weight distribution map is used to describe the position weight of each pixel in the virtual image. For the overlapping area between multiple virtual images, the position weight of any pixel is higher the closer it is to the center of the virtual image.

[0059] In one possible implementation, the multi-path corrected images around the vehicle include a corrected image of the left side of the vehicle, a corrected image of the right side of the vehicle, and a corrected image of the rear side of the vehicle. The stitching module 403 is used to increase the position weight of pixels belonging to the diagonal region in the virtual image corresponding to the corrected image of the rear side of the vehicle for the diagonal region of the simulated observation view. It calculates the pixel value of the first pixel in the complete image around the vehicle based on the pixel value and position weight of each first pixel in the overlapping area of ​​the image in different virtual images. It uses the pixel value of each second pixel in the non-overlapping area of ​​the image in the virtual image as the pixel value of the second pixel in the complete image around the vehicle. It generates a complete image around the vehicle based on the pixel values ​​of the first pixel in the complete image around the vehicle and the pixel values ​​of the second pixel in the complete image around the vehicle.

[0060] In one possible implementation, the device further includes: a second acquisition module for acquiring the risk level of the vehicle's blind spot and the on / off state of the turn signal, the on / off state of the turn signal including an on state or a off state; and an indication module for indicating that the vehicle's operating conditions meet the triggering conditions for displaying the blind spot image in response to the risk level of the vehicle's blind spot being greater than a level threshold or the turn signal switching from an off state to an on state.

[0061] In one possible implementation, the device further includes: a third acquisition module for acquiring multiple original images of the vehicle's surroundings; and a processing module for performing distortion correction processing on the multiple original images to obtain multiple corrected images.

[0062] This device acquires multiple corrected images of the vehicle's surroundings and converts them into multiple virtual images of the vehicle from a simulated observation perspective. This facilitates subsequent stitching of images from different angles within the same viewpoint. Each virtual image is then stitched together to obtain a complete image of the vehicle's surroundings. Furthermore, by monitoring the vehicle's operating conditions, it determines whether the conditions meet the triggering conditions for blind spot image display. When these conditions are met, the blind spot portion of the complete image surrounding the vehicle is displayed, ensuring that the blind spot image is only triggered when needed, thus avoiding driver distraction. This allows for the real-time display of a complete blind spot image using existing vehicle equipment when the driver requires blind spot monitoring. This reduces blind spot safety hazards during vehicle operation and improves overall driving safety without increasing manufacturing costs or after-sales maintenance complexity.

[0063] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0064] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-described blind spot monitoring methods.

[0065] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0066] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the blind spot monitoring methods described above.

[0067] In an exemplary embodiment, a vehicle is also provided, the vehicle including: a vehicle body; a cockpit domain controller, the cockpit domain controller being used to perform any of the above-described blind spot monitoring methods on the vehicle body.

[0068] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the vehicle operating conditions, multi-channel calibrated images around the vehicle, multiple virtual images, and complete images around the vehicle involved in this application were all obtained with full authorization.

[0069] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0070] It should be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0071] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for blind spot monitoring, characterized in that, The method includes: Acquire multi-channel corrected images of the area surrounding the vehicle; The perspective of the multi-channel corrected images is converted to the simulated observation perspective to obtain multiple virtual images of the vehicle under the simulated observation perspective; Each virtual image is stitched together to obtain a complete image of the area around the vehicle; In response to the vehicle operating conditions meeting the triggering conditions for blind spot image display, the blind spot portion of the complete image around the vehicle is displayed.

2. The method according to claim 1, characterized in that, The step of converting the viewpoint of the multi-channel corrected images to a simulated observation viewpoint to obtain multiple virtual images of the vehicle under the simulated observation viewpoint includes: By performing viewpoint transformation on each pixel in the multi-path corrected image using the homography matrix, multiple virtual images of the vehicle under the simulated observation viewpoint are obtained.

3. The method according to claim 2, characterized in that, Before converting the viewpoint of the multi-path corrected image to the simulated observation viewpoint, the method further includes: The information of the virtual camera and the vehicle-mounted surround view camera corresponding to the simulated observation perspective are obtained. The information of the virtual camera includes a first internal parameter and a first external parameter, and the information of the vehicle-mounted surround view camera includes a second internal parameter and a second external parameter. The homography matrix corresponding to the virtual camera and the vehicle surround view camera is calculated based on the first internal parameter, the first external parameter, the second internal parameter, and the second external parameter. The homography matrix is ​​used to switch the viewpoint from the multi-channel corrected image to the simulated observation viewpoint through the viewpoint conversion between the cameras.

4. The method according to any one of claims 1-3, characterized in that, Before stitching together each virtual image, the method further includes: Obtain the weight distribution map corresponding to each virtual image. The weight distribution map is used to describe the position weight of each pixel in the virtual image. For the overlapping area between multiple virtual images, the position weight of any pixel is higher the closer it is to the center of the virtual image.

5. The method according to claim 4, characterized in that, The multi-channel corrected images around the vehicle include corrected images of the left side of the vehicle, the right side of the vehicle, and the rear side of the vehicle. The process of stitching together each virtual image to obtain a complete image of the area around the vehicle includes: For the diagonal region of the simulated observation viewpoint, increase the positional weight of the pixels belonging to the diagonal region in the virtual image corresponding to the corrected image of the rear of the vehicle; Calculate the pixel value of the first pixel in the complete image around the vehicle based on the pixel value and position weight of each first pixel in the overlapping area of ​​the image in different virtual images; The pixel value of each second pixel in the non-overlapping area of ​​the image is used as the pixel value of the second pixel in the complete image around the vehicle. A complete image of the vehicle's surroundings is generated based on the pixel values ​​of the first pixel in the complete image of the vehicle's surroundings and the pixel values ​​of the second pixel in the complete image of the vehicle's surroundings.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: The risk level of the vehicle's blind spot and the on / off status of the turn signals are obtained, including the on or off status of the turn signals. In response to the risk level of the vehicle blind spot being greater than a level threshold or the turn signal switching from the off state to the on state, the vehicle operating condition is indicated to meet the trigger condition for displaying the blind spot image.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Acquire multiple raw images of the area surrounding the vehicle; The original multi-channel images are subjected to distortion correction processing to obtain the corrected multi-channel images.

8. A device for blind zone monitoring, characterized in that, The method includes: The first acquisition module is used to acquire multi-channel corrected images around the vehicle; The conversion module is used to convert the perspective of the multi-channel corrected images to the simulated observation perspective, so as to obtain multiple virtual images of the vehicle under the simulated observation perspective; The stitching module is used to stitch together each virtual image to obtain a complete image of the area around the vehicle. The display module is used to display the blind spot portion of the complete image around the vehicle in response to the triggering condition that the vehicle's operating conditions meet the blind spot image display requirement.

9. A non-transitory computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the blind spot monitoring method as described in any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicles include: Vehicle body; A cockpit domain controller for performing the blind spot monitoring method of any one of claims 1 to 7 on the vehicle body.

Images