Blind area display method and device, electronic equipment and readable storage medium

Abstract

The application discloses a blind area display method and device, electronic equipment and a readable storage medium, and relates to the technical field of display. The method comprises the following steps: collecting a first image and a second image based on a mixed reality device worn by a target person who is on a target vehicle; determining a deformation parameter based on eye pose information, position information of an image collection device and a shielding object in the target vehicle, wherein the shielding object is an object that shields the field of view of the target person to form a field of view blind area; adjusting the second image based on the deformation parameter to obtain a corrected image; synthesizing the first image and the corrected image to replace a local image corresponding to the field of view blind area in the first image with a local image corresponding to the field of view blind area in the corrected image to obtain a target image; and displaying the target image in the mixed reality device. The target image displayed to the target person is obtained by image processing on the collected first image and second image, and the picture fragmentation of the target image is reduced.

Description

Technical Field

[0001] The present application relates to the field of display technology, and more specifically, to a blind area display method, device, electronic device, and readable storage medium. Background Art

[0002] Currently, vehicles can have obstructing objects that block the view of occupants, creating blind spots. While sensors can capture images of these blind spots and display them to occupants, these images often produce a fragmented and poorly rendered image, resulting in a poor user experience. Summary of the Invention

[0003] The present application proposes a blind spot display method, device, electronic device and readable storage medium.

[0004] In a first aspect, an embodiment of the present application provides a blind spot display method, the method comprising: acquiring a first image based on a mixed reality device worn by a target person riding in a target vehicle, wherein the first image is an image corresponding to the eye posture information of the target person; acquiring a second image based on an image acquisition device arranged in the target vehicle, wherein the second image is a real-scene image corresponding to the blind spot of the target person's field of vision in the surrounding environment of the target vehicle; determining deformation parameters based on the eye posture information, position information of the image acquisition device, and an obstructing object in the target vehicle, wherein the obstructing object is an object that obstructs the field of vision of the target person to form the blind spot of vision; adjusting the second image based on the deformation parameters to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot of vision in the first image; synthesizing the first image and the corrected image to replace the local image corresponding to the blind spot of vision in the first image with the local image corresponding to the blind spot of vision in the corrected image to obtain a target image; and displaying the target image in the mixed reality device.

[0005] Optionally, for a possible implementation, the first image and the corrected image are synthesized to replace the local image corresponding to the blind area of ​​the field of view in the first image with the local image corresponding to the blind area of ​​the field of view in the corrected image to obtain a target image, including: shielding the local image corresponding to the blind area of ​​the field of view in the first image to obtain an alternative image; superimposing the corrected image on the local image corresponding to the shielded blind area of ​​the field of view in the alternative image, and splicing it with the alternative image to obtain the target image.

[0006] Optionally, for a possible implementation, the superimposing the modified image to the local image corresponding to the shielded visual field blind area in the alternative image, and splicing the alternative image to obtain the target image, comprises: adjusting at least one of a first specified parameter of the alternative image and a second specified parameter of the modified image, so that the first specified parameter is consistent with the second specified parameter, wherein the first specified parameter and the second specified parameter each include at least one of exposure, sharpness, chromatic aberration, and color tone; after the at least one of the first specified parameter and the second specified parameter is adjusted, superimposing the modified image to the local image corresponding to the shielded visual field blind area in the alternative image, and splicing the alternative image to obtain the target image.

[0007] Optionally, for a possible implementation, the superimposing the modified image to the local image corresponding to the shielded visual field blind area in the alternative image, and splicing the alternative image to obtain the target image, comprises: superimposing the modified image to the local image corresponding to the shielded visual field blind area in the alternative image to obtain an intermediate image; determining a splicing area in the intermediate image; performing image smoothing processing on the splicing area to obtain the target image.

[0008] Optionally, for a possible implementation, the displaying the target image in the mixed reality device comprises: identifying the target image; in a case where it is identified that there is a target object in the target image, generating prompt information and providing the prompt information to a target person; and displaying the target image in the mixed reality device.

[0009] Optionally, for a possible implementation, after the displaying the target image in the mixed reality device, the method further comprises: obtaining current state information of the target vehicle, the state information comprising at least one of vehicle speed information, tire pressure information, and navigation information; and displaying the state information in the mixed reality device.

[0010] In a second aspect, an embodiment of the present application further provides a blind spot display device, which includes: a first acquisition unit, a second acquisition unit, a deformation parameter determination unit, an adjustment unit, a processing unit, and a display unit. Among them, the first acquisition unit is used to acquire a first image based on a mixed reality device worn by a target person riding in a target vehicle, wherein the first image is an image corresponding to the eye posture information of the target person; the second acquisition unit is used to acquire a second image based on an image acquisition device set in the target vehicle, wherein the second image is a real scene image corresponding to the blind spot of the target person in the surrounding environment of the target vehicle; the deformation parameter determination unit is used to determine the deformation parameter based on the eye posture information, the position information of the image acquisition device and the blocking object in the target vehicle, wherein the blocking object is an object that blocks the field of view of the target person to form the blind spot; the adjustment unit is used to adjust the second image based on the deformation parameter to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image; the processing unit is used to synthesize the first image and the corrected image to replace the local image corresponding to the blind spot in the first image with the local image corresponding to the blind spot in the corrected image to obtain the target image; the display unit is used to display the target image on the mixed reality device.

[0011] In a third aspect, an embodiment of the present application further provides an electronic device comprising: one or more processors; a memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method described in the first aspect.

[0012] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which program code is stored. The program code can be called by a processor to execute the method described in the first aspect above.

[0013] The blind spot display method, device, electronic device and readable storage medium provided by the embodiments of the present application first capture a first image based on a mixed reality device worn by a target person riding in a target vehicle; capture a second image based on an image capture device set in the target vehicle; then determine deformation parameters based on eye posture information, position information of the image capture device and an obstructing object in the target vehicle, where the obstructing object is an object that blocks the field of view of the target person and forms a blind spot; adjust the second image based on the deformation parameters to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image; synthesize the first image and the corrected image to replace the local image corresponding to the blind spot in the first image with the local image corresponding to the blind spot in the corrected image to obtain a target image; thereby, displaying the target image in the mixed reality device. Since the captured image and the image observed by the target person through the mixed reality device are two types of images, if the captured real-life image of the blind spot and the observed image are directly processed, if the target person's eye posture information changes, and there is an error between the speed of the graphics processing and the speed of the driver's glasses shaking, the image obtained after image processing will have a strong sense of disconnection. In the embodiment of the present application, the first image and the second image are both captured images, wherein the first image is an image corresponding to the eye posture information of the target person; the second image is a real-life image corresponding to the blind spot of the target person's field of vision in the surrounding environment of the target vehicle; thus, image processing is performed based on the second image, and the target image obtained is obtained by processing the two captured images. Moreover, precisely because the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the field of vision in the first image, the display effect of the acquired target image can be improved and the sense of disconnection of the target image can be reduced.

[0014] Other features and advantages of the embodiments of the present application will be described in the following description and, in part, will become apparent from the description or be understood by practicing the embodiments of the present application. The objectives and other advantages of the embodiments of the present application can be achieved and obtained through the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A diagram showing an application scenario of the blind spot display method provided in an embodiment of the present application is shown;

[0017] Figure 2 A flowchart of a blind spot display method provided by an embodiment of the present application is shown;

[0018] Figure 3 A schematic diagram showing a blind spot in the field of view provided by an embodiment of the present application is shown;

[0019] Figure 4 A schematic diagram showing a partial image corresponding to a blind area of ​​the visual field in a first image provided by an embodiment of the present application is shown;

[0020] Figure 5 A schematic diagram of determining deformation parameters provided by an embodiment of the present application is shown;

[0021] Figure 6 A schematic diagram of a corrected image provided by an embodiment of the present application is shown;

[0022] Figure 7 A flowchart of a blind spot display method according to another embodiment of the present application is shown;

[0023] Figure 8 A schematic diagram showing alternative images provided by an embodiment of the present application is shown;

[0024] Figure 9 A schematic diagram of a target image provided by an embodiment of the present application is shown;

[0025] Figure 10 A flowchart of a blind spot display method provided by another embodiment of the present application is shown;

[0026] Figure 11 A structural block diagram of a blind spot display device provided in an embodiment of the present application is shown;

[0027] Figure 12 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0028] Figure 13 A structural block diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0031] Currently, vehicles can have obstructing objects, which can block the view of occupants, creating blind spots. While sensors can capture images of these blind spots and display them to occupants, these images often exhibit a strong sense of fragmentation, resulting in poor display quality and a poor user experience. Reducing this fragmentation and improving the display quality of blind spots is an urgent issue.

[0032] Currently, real-scene images of blind spots can be collected and then displayed to the driver.

[0033] However, the inventors discovered during their research that since the real-scene image collected and the picture directly observed by the target person are two types of images, if the real-scene image collected in the blind spot and the observed picture are directly processed, if the eye posture information of the target person changes at this time, and there is an error between the speed of graphic processing and the shaking speed of the driver's glasses, the image obtained after image processing will have a strong sense of picture fragmentation, that is, the display effect of the obtained image will be poor.

[0034] Therefore, in order to solve or partially solve the above problems, the embodiments of the present application provide a blind spot display method, device, electronic device and readable storage medium.

[0035] See also Figure 1 , Figure 1 An application scenario diagram of the blind spot display method provided in an embodiment of the present application, namely, a blind spot display scenario 100 is shown. Figure 1: A blind spot display scene 100 is shown in the figure, which includes a target vehicle 110 and a pedestrian 120, and the pedestrian 120 is located at the zebra crossing 190 and is passing normally. The target person in the target vehicle 110 will have a blind spot 180 when observed by the target vehicle, wherein the target person may be the driver of the target vehicle 110. At this time, the pedestrian 120 is located in the blind spot 180, and the driver of the target vehicle 110 is unable to observe the pedestrian 120, which may cause a safety accident. Therefore, the blind spot display method provided in the embodiment of the present application can be used to display the real scene corresponding to the blind spot 180 to the driver, thereby helping to improve the driver's cognitive ability of the environment around the target vehicle 110, effectively reducing traffic accidents caused by the blind spot 180, and improving driving safety. For a detailed introduction to the blind spot display method, please refer to the subsequent method embodiments.

[0036] See also Figure 2 , Figure 2 A method flow chart of a blind area display method provided by an embodiment of the present application is shown. The blind area display method can be applied to Figure 1 In the blind spot display scenario shown in , the method may specifically include steps S110 to S160.

[0037] Step S110: Based on the mixed reality device worn by the target person riding in the target vehicle, a first image is captured, wherein the first image is an image corresponding to the eye posture information of the target person.

[0038] The target vehicle is a car, which can be a traditional gasoline car or a new energy car, and the embodiments of this application do not specifically limit this. The target person in the target vehicle can be the driver of the target vehicle or a passenger in the target vehicle, such as a passenger in the front passenger seat or a passenger in the back seat. It should be noted that the subsequent description will be specifically explained using the example of the target person being the driver of the target vehicle.

[0039] It is understandable that the target person has a field of view, and as the target person's head rotates, the image corresponding to the field of view will change. The field of view can be determined by the target person's eye posture information. Among them, the eye posture information can specifically include eye position information and eye posture information. In some embodiments, the position information can be represented by the position of the target person's eyes in the vehicle coordinate system, and the posture information can be represented by the target person's roll angle, pitch angle and heading angle. Among them, the vehicle coordinate system is a coordinate system established based on the target vehicle.

[0040] With the development of technologies related to mixed reality (MR) devices, the weight, battery life, and performance of MR devices are becoming increasingly sophisticated. Therefore, in some embodiments, the target person can wear a MR device. This allows the MR device to capture a first image, and when the target image is subsequently processed to obtain the target image, the fragmented appearance of the target image can be reduced, thereby improving the display quality of the target image.

[0041] The first image is an image corresponding to the eye posture information of the target person. In other words, the first image corresponds to the field of view of the target person's eyes. In some embodiments, the mixed reality device may be configured with a camera, so that the camera configured for the mixed reality device can capture images to obtain the first image.

[0042] For example, the field of view of a camera configured in the mixed reality device can be pre-set so that the field of view is the same as or approximately the same as the field of view of the target person's eyes. Thus, the image captured by the set camera can be used as the first image.

[0043] In another exemplary embodiment, a camera configured in a mixed reality device can capture images over a larger range, and then crop the captured images to obtain an image corresponding to the field of view of the target person's eyes as the first image. Cropping the images to obtain the image corresponding to the field of view of the target person's eyes can be processed using a pre-trained cropping model, which can be a model trained based on a deep neural network.

[0044] It should be noted that mixed reality devices are generally equipped with lenses. However, if the target person directly observes the real world through the lenses of the mixed reality device, the target image displayed in the mixed reality device will appear disconnected. Therefore, in the embodiments provided in this application, the lenses configured for the mixed reality device can be opaque lenses, and the content to be displayed can be projected onto these lenses.

[0045] Step S120: Based on the image acquisition device provided on the target vehicle, a second image is acquired, wherein the second image is a real scene image corresponding to the blind spot of the target person's field of view in the surrounding environment of the target vehicle.

[0046] It is understandable that there may be obstructing objects in the target vehicle, and the target person's line of sight passes through the obstructing objects in the target vehicle, which will form a blind spot. For example, please refer to Figure 3 , Figure 3 A schematic diagram showing a blind area of ​​vision in an embodiment of the present application is shown. Figure 31 shows a target vehicle 110 and a target person's eyes 310. There is an obstructing object 111 in the target vehicle 110, so that the human eye 310 and the obstructing object 111 form a blind spot 180. Figure 3 The blocking object 111 shown in FIG. 1 is the A-pillar on the left side of the target vehicle.

[0047] Therefore, it is difficult for the target person in the target vehicle to directly observe the real scene in the blind spot from within the target vehicle, which may lead to the risk of a traffic accident. In order to subsequently display the target image including the real scene image of the blind spot through the mixed reality device, the real scene image corresponding to the blind spot can be first obtained.

[0048] In some embodiments, the target vehicle may be provided with an image acquisition device, such as a camera. Thus, an initial image of the target vehicle's surroundings may be first acquired, and then an image corresponding to the blind spot may be cropped from the initial image as the real scene image corresponding to the blind spot, i.e., the second image. For example, please continue to refer to Figure 3 , Figure 3 A camera 112 is provided on the outer side of the vehicle body of the target vehicle 110 , so that the camera 112 can collect an initial image around the target vehicle.

[0049] It should be noted that the number of cameras installed on the target vehicle can be one or more. In some embodiments, images captured by multiple cameras can be fused to obtain an initial image with a wider field of view. It is understood that the initial image captured by the camera installed on the target vehicle should at least include a real-life image corresponding to the blind spot. Exemplarily, the camera can be installed on the body of the target vehicle.

[0050] Step S130: Determine deformation parameters based on the eye posture information, the position information of the image acquisition device, and the blocking object in the target vehicle, where the blocking object is an object that blocks the field of view of the target person and forms the blind spot.

[0051] Step S140: adjusting the second image based on the deformation parameter to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image.

[0052] As can be seen from the preceding description, the first image corresponds to the target person's eye posture information. As the target person's line of sight passes through obstructing objects in the target vehicle, a blind spot is formed. Therefore, a partial image corresponding to the blind spot can be determined in the first image.

[0053] For example, see Figure 4 , Figure 4 A schematic diagram of a partial image corresponding to a blind area in the first image provided by an embodiment of the present application is shown. Figure 4 The first image 410 is shown in FIG. The first image includes a target vehicle 110. There is an obstructing object 111 in the target vehicle 110, and the obstructing object 111 will form a blind spot in the field of view. Figure 4 In the first image 410 shown in FIG, it can be determined that the local image corresponding to the blind spot formed by the blocking object 111 is the local image 411 corresponding to the blind spot in the first image. Figure 4 The local image 411 corresponding to the blind area of ​​the visual field in the first image shown in FIG. 4 is represented by a dotted frame in the figure.

[0054] In some embodiments, the first image can be recognized using a pre-trained recognition model to determine the local image of the blind spot in the first image. The recognition model can be a deep neural network model. Thus, the first image acquired through the aforementioned steps can be used as input to the recognition model, and the output of the recognition model can be obtained as the local image corresponding to the blind spot in the first image.

[0055] In other embodiments, a designated type of object may be pre-set. After acquiring a first image, the first image is identified and the image region containing the designated type of object is determined. The image region containing the designated type of object is then used as the partial image of the blind spot in the first image. For example, if the target person is a driver, the driver may pre-set the A-pillar of the target vehicle as the designated type of object.

[0056] After determining the local image corresponding to the blind area in the first image, the local image corresponding to the blind area in the first image may be further processed using the second image to obtain a target image.

[0057] In some embodiments, the area corresponding to the partial image corresponding to the blind spot in the first image may be first masked, and then the first image with the masked portion of the image area may be spliced ​​with the second image to obtain the target image. Masking the area corresponding to the partial image corresponding to the blind spot in the first image may include setting the transparency of the area corresponding to the partial image corresponding to the blind spot in the first image to 100%, or filling the area corresponding to the partial image corresponding to the blind spot in the first image with a preset background color, such as white, black, etc., which is not specifically limited in the embodiments of the present application.

[0058] In some other embodiments, the second image may be first adjusted, for example, by adjusting its size, to obtain a corrected image, and then the first image and the corrected image may be synthesized to obtain the target image. Detailed descriptions can be found in the subsequent steps.

[0059] For some other embodiments, the second image may be directly placed on the area corresponding to the local image corresponding to the blind spot in the first image, and the area corresponding to the local image corresponding to the blind spot in the first image may be filled and covered by the second image.

[0060] As can be seen from the above introduction, the eye posture information can specifically include the position information of the eyes and the posture information of the eyes. For some embodiments, the eye posture information can be obtained by a mixed reality device. Among them, the mixed reality device can be configured with a camera and a gyroscope, so that the mixed reality device can realize simultaneous positioning and mapping (SLAM) by the camera and in combination with the gyroscope, and then determine the position of the mixed reality device in the vehicle coordinate system, and further obtain the eye posture information of the target person wearing the mixed reality device. It is understandable that the determined eye posture information is characterized in the vehicle coordinate system. Exemplarily, simultaneous positioning and mapping can be realized by the Structure from Motion (SFM) algorithm, and specifically COLMAP (COLLISION-MAPPING) software can be utilized to realize simultaneous positioning and mapping, which is not specifically limited in the embodiments of the present application.

[0061] As a result, the eye posture information of the target person determined by combining the camera and gyroscope in the mixed reality device has a high degree of accuracy. Furthermore, because the eye posture information is collected directly from the mixed reality device worn by the target person, it does not require extensive conversion and has a low computational complexity.

[0062] Furthermore, since the second image is acquired through the image acquisition device of the target vehicle, the initial image around the target vehicle can be acquired first, and then the image corresponding to the blind spot can be cropped out of the initial image as the real scene image corresponding to the blind spot, that is, the second image. Therefore, if the position of the image acquisition device set on the target vehicle changes, the acquired initial image will change, and the size of the cropped second image will also change. Therefore, in order to improve the display effect of the acquired target image and reduce the sense of fragmentation of the target image, the second image can also be adjusted, and then the target image can be obtained based on the adjusted second image. For example, the adjustment can be a size adjustment.

[0063] Furthermore, since it is necessary to perform image processing on the local image corresponding to the blind spot in the first image based on the adjusted second image, wherein the blind spot is an area formed by an obstructing object in the target vehicle blocking the field of view of the target person, and the obstructing object is an object that blocks the field of view of the target person to form the blind spot. Therefore, for some embodiments, the deformation parameters can be first determined based on the eye posture information, the position information of the image acquisition device, and the obstructing object in the target vehicle. Then, the second image is adjusted based on the deformation parameters to obtain a corrected image. Exemplarily, the deformation parameters may include geometric scaling coefficients, distortion correction parameters, or rotation parameters, etc. Among them, the deformation parameters can be determined based on the principle of perspective projection. The position of the obstructing object in the vehicle and the shape and size of the obstructing object will affect the specific value of the obtained deformation parameters.

[0064] It can be understood that the second image after the deformation parameters are adjusted is the corrected image. The corrected image can be subsequently used to perform image processing on the local image corresponding to the blind spot in the first image, for example, to synthesize the first image and the corrected image. In the embodiment provided in the present application, the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image. It should be noted that the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image, and the size of the corrected image can be close to the size of the local image corresponding to the blind spot in the first image, or the size of the corrected image can be smaller than the size of the local image corresponding to the blind spot in the first image. And precisely because the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image, the display effect of the acquired target image can be improved and the sense of fragmentation of the target image can be reduced.

[0065] For example, see Figure 5 , Figure 5 A schematic diagram of determining deformation parameters provided in an embodiment of the present application is shown. Figure 5 Figure 5 shows the target person's eyes 510, an obstructing object 520, a real-life image 530, a camera 540, and a second image 550. When the target person's eyes 510 are obstructed by the obstructing object 520, a blind spot is formed in the area corresponding to the real-life image 530. The obstructing object 520 forms a partial image corresponding to the blind spot in the first image. The camera 540 can be installed on the target vehicle, so that the real-life image 530 can be captured by the camera 540 and further processed, such as by cropping, to obtain the second image 550.

[0066] pass Figure 5It can be seen that the second image 550 obtained after being captured and processed by the camera 540 has a size difference with the blocking object 520. Therefore, the deformation parameters can be determined through the principle of perspective projection, and the second image 550 can be adjusted according to the deformation parameters to obtain a corrected image, so that the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the field of view in the first image.

[0067] See also Figure 6 , Figure 6 Schematic diagram of the corrected image provided by the embodiment of the present application is shown. Figure 5 The camera 540 shown in the figure acquires the real scene image 530 to obtain the second image 550, and then adjusts the second image 550 according to the deformation parameter.

[0068] Step S150: synthesizing the first image and the corrected image to replace the local image corresponding to the blind spot in the first image with the local image corresponding to the blind spot in the corrected image to obtain a target image.

[0069] After obtaining the corrected image, image processing can be performed on the partial image corresponding to the blind spot in the first image based on the corrected image to obtain a target image. Specifically, the first image and the corrected image can be synthesized to replace the partial image corresponding to the blind spot in the first image with the partial image corresponding to the blind spot in the corrected image to obtain the target image.

[0070] The first image and the corrected image are synthesized to obtain the target image. This can be done by first shielding the area corresponding to the local image corresponding to the blind spot in the field of view in the first image, and then splicing the first image with the shielded image area with the corrected image to obtain the target image.

[0071] The mixed reality device can communicate with the target vehicle to obtain the corrected image generated in the above steps, for example, by communicating with the target vehicle's on-board system. The mixed reality device then acquires the first image through its own camera, and further synthesizes the first image and the corrected image to obtain the target image. The corrected image is obtained by adjusting the second image based on the deformation parameters. Therefore, in order to ensure the temporal synchronization of the first image and the corrected image, the bit rate for acquiring the first image and the bit rate for acquiring the second image can be set to ensure that the bit rate of the first image is the same as the bit rate of the second image, thereby ensuring the temporal synchronization of the first image and the second image.

[0072] Step S160: Displaying the target image in the mixed reality device.

[0073] Therefore, the target images obtained in the above steps are all obtained by image processing the first image and the second image obtained by acquisition. Therefore, even if the eye posture information of the target person changes, for example, the head of the target person rotates, it can still be ensured that the target image generated according to the above method has a good effect, reducing the sense of fragmentation of the target image.

[0074] In some embodiments, the acquired target image can be displayed to the target user through a mixed reality device. Since the target image has a low sense of fragmentation, the display effect can be improved to a certain extent, thereby enhancing the target user's experience.

[0075] In other implementations, the acquired target image can also be adjusted to specific parameters, thereby further improving the display quality of the target image and reducing the sense of fragmentation. The adjusted target image can then be displayed to the target user via a mixed reality device. For a detailed description, please refer to the subsequent embodiments.

[0076] Displaying the target image through the mixed reality device may be projecting the target image onto a lens of the mixed display device.

[0077] The blind spot display method, device, electronic device and readable storage medium provided by the embodiments of the present application first capture a first image based on a mixed reality device worn by a target person riding in a target vehicle; capture a second image based on an image capture device set in the target vehicle; then determine deformation parameters based on eye posture information, position information of the image capture device and an obstructing object in the target vehicle, where the obstructing object is an object that blocks the field of view of the target person and forms a blind spot; adjust the second image based on the deformation parameters to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the first image; synthesize the first image and the corrected image to replace the local image corresponding to the blind spot in the first image with the local image corresponding to the blind spot in the corrected image to obtain a target image; thereby, displaying the target image in the mixed reality device. Since the captured image and the image observed by the target person through the mixed reality device are two different types of images, if the captured real-life image of the blind spot and the observed image are directly processed, if the target person's eye posture information changes, and there is a discrepancy between the speed of the image processing and the speed of the driver's glasses shaking, the image obtained after image processing will have a strong sense of disconnection. In the embodiment of the present application, the first image and the second image are both captured images, wherein the first image is an image corresponding to the eye posture information of the target person; the second image is a real-life image corresponding to the blind spot of the target person's field of vision in the environment surrounding the target vehicle; thus, image processing is performed based on the second image, and the target image obtained is obtained by processing the two captured images. Moreover, precisely because the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the field of vision in the first image, the display effect of the acquired target image can be improved and the sense of disconnection of the target image can be reduced.

[0078] See also Figure 7 , Figure 7 A method flow chart of a blind area display method provided by an embodiment of the present application is shown. The blind area display method can be applied to Figure 1 In the blind spot display scenario shown in , the method may specifically include steps S210 to S270.

[0079] Step S210: Based on the mixed reality device worn by the target person riding in the target vehicle, a first image is captured, wherein the first image is an image corresponding to the eye posture information of the target person.

[0080] Step S220: Based on the image acquisition device provided on the target vehicle, a second image is acquired, wherein the second image is a real scene image corresponding to the blind spot of the target person's field of vision in the surrounding environment of the target vehicle.

[0081] Step S230: Determine deformation parameters based on the eye posture information, the position information of the image acquisition device, and the blocking object in the target vehicle, where the blocking object is an object that blocks the field of view of the target person and forms the blind spot.

[0082] Step S240: adjusting the second image based on the deformation parameter to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the field of view in the first image.

[0083] Among them, steps S210 to S240 have been described in detail in the above embodiments and will not be repeated here.

[0084] Step S250: shielding the local image corresponding to the blind area of ​​the visual field in the first image to obtain an alternative image.

[0085] Step S260: superimposing the corrected image onto the partial image corresponding to the blocked blind area of ​​the visual field in the candidate image, and splicing it with the candidate image to obtain the target image.

[0086] First, the partial image corresponding to the blind spot in the first image can be masked to obtain an alternative image. The transparency of the area corresponding to the partial image in the first image corresponding to the blind spot can be set to 100%, or the area corresponding to the partial image in the first image corresponding to the blind spot can be filled with a preset background color, such as white or black.

[0087] For example, please see Figure 4 , you can Figure 4 The partial image 411 corresponding to the blind area in the first image 410 is shielded. Figure 8 , Figure 8 A schematic diagram of an alternative image provided in an embodiment of the present application is shown. Figure 8 The alternative image 810 is shown in FIG. 8 , which includes a partial image 820 corresponding to the blind area in the first image. The partial image 820 corresponding to the blind area in the first image is the partial image corresponding to the blind area that needs to be blocked. Therefore, the corrected image can be superimposed on the partial image corresponding to the blocked blind area in the alternative image, and the target image can be obtained by splicing it with the alternative image. It should be noted that Figure 8 The local image 820 corresponding to the blind area of ​​the field of view in the first image shown in FIG can be marked by a dotted frame.

[0088] For further information, please refer to Figure 9 , Figure 9 A schematic diagram of a target image provided by an embodiment of the present application is shown. Figure 9The target image 910 is shown in FIG. Figure 7 The modified image 710 is shown superimposed on Figure 8 The partial image 820 corresponding to the blocked visual field blind area in the alternative image 820 shown in FIG. 8 is used to realize the correction image 710 and the correction image 710. Figure 8 The candidate images 810 shown are stitched together to obtain the target image 910. It should be noted that Figure 7 The corrected image shown in Figure 4 The image corresponding to the partial image 411 corresponding to the blind spot in the first image on the left side of the first image 410 in FIG. Figure 9 In the figure, the corrected image 710 is superimposed on Figure 8 The local image 820 corresponding to the blind area of ​​the field of view in the first image on the left side.

[0089] Optionally, since the first image is obtained by processing an image captured by a camera of a mixed reality device, and the second image is obtained by processing an image captured by a camera installed on a target vehicle, the alternative image determined based on the first image and the corrected image determined based on the second image are obtained based on images captured by different cameras. Furthermore, there may be differences between the first specified parameters of the alternative image and the second specified parameters of the corrected image. The first specified parameters and the second specified parameters both include at least one of exposure, sharpness, color difference, and hue. It is understandable that if there is a difference between the first specified parameters and the second specified parameters, the corrected image is superimposed on the local image corresponding to the shielded blind spot in the alternative image, and is spliced ​​with the alternative image to obtain the target image. The target image will appear more abrupt, which can easily lead to the problem of unnatural display effects.

[0090] Therefore, in some embodiments, step S260 may further include step S261 and step S262.

[0091] Step S261: Adjust at least one of the first specified parameter of the alternative image and the second specified parameter of the corrected image so that the first specified parameter is consistent with the second specified parameter, wherein the first specified parameter and the second specified parameter both include at least one of exposure, sharpness, color difference and hue.

[0092] Step S262: After at least one of the first specified parameter and the second specified parameter is adjusted, the corrected image is superimposed on the local image corresponding to the blocked blind area of ​​the field of view in the alternative image, and is spliced ​​with the alternative image to obtain the target image.

[0093] For example, the first specified parameter is exposure and sharpness, and the second specified parameter also includes exposure and sharpness. For another example, the first specified parameter is exposure, color and sharpness, and the second specified parameter also includes exposure, color and sharpness. The image automatic compensation technology is a method of adjusting the specified parameter of the image by using an algorithm.

[0094] It can be understood that the first specified parameter of the candidate image can be adjusted only, the second specified parameter of the modified image can be adjusted only, or the first specified parameter and the second specified parameter can be adjusted. In the embodiments provided in the present application, the first specified parameter and the second specified parameter are consistent after at least one of the first specified parameter and the second specified parameter is adjusted. At least one of the first specified parameter and the second specified parameter of the candidate image and the modified image can be adjusted, and at least one of the parameters including exposure, sharpness, color difference and color tone can be adjusted.

[0095] Further, the modified image can be superimposed on the local image corresponding to the visual field blind area of the candidate image which is shielded after at least one of the first specified parameter and the second specified parameter is adjusted, and the target image can be obtained by splicing the candidate image. The splicing method can be referred to the foregoing description, and will not be described here.

[0096] Optionally, for some embodiments, the candidate image and the modified image can be spliced to obtain the target image first, and then the specified parameter of the target image can be adjusted to make the consistency of the specified parameter of the target image better.

[0097] In the embodiments provided in the present application, the first specified parameter and the second specified parameter are consistent after at least one of the first specified parameter and the second specified parameter is adjusted. Therefore, it can be ensured that the target image obtained based on the modified image and the candidate image will not have a split feeling, and the display effect of the obtained target image is improved.

[0098] Optionally, since the target image can be regarded as two different image spliced images, the splicing area of the two different images is prone to have unnatural image transition, so that the display effect of the target image is poor. Therefore, in some embodiments, step S260 can further include steps S264 to S266.

[0099] Step S264: superimposing the corrected image onto the partial image corresponding to the shielded blind area of ​​the visual field in the candidate image, and splicing it with the candidate image to obtain an intermediate image.

[0100] Step S265: Determine a stitching area in the intermediate image.

[0101] Step S266: performing image smoothing processing on the stitching area to obtain the target image.

[0102] First, the corrected image can be superimposed on the partial image corresponding to the blocked blind spot in the candidate image, and then stitched together with the candidate image to obtain an intermediate image. This is similar to the method described in the previous step of superimposing the corrected image on the partial image corresponding to the blocked blind spot in the candidate image and stitching it together with the candidate image to obtain the target image, and will not be repeated here.

[0103] It should be noted that the corrected image can be directly superimposed on the local image corresponding to the shielded blind spot in the alternative image, and spliced ​​with the alternative image to obtain an intermediate image; or similarly to the above steps, the first specified parameters of the alternative image and the second specified parameters of the corrected image can be adjusted respectively, and then the adjusted corrected image can be superimposed on the local image corresponding to the shielded blind spot in the adjusted alternative image, and spliced ​​with the adjusted alternative image to obtain the intermediate image.

[0104] Furthermore, the stitching area can be determined in the obtained intermediate image. Exemplarily, the stitching area can be an area range centered on the pixel point at the stitching of the corrected image and the alternative image. Specifically, for example, all the pixel points at the stitching of the corrected image and the alternative image can be determined first, and then each pixel point at the stitching and all the pixel points adjacent to each pixel point are determined as stitching pixel points, and then all the stitching pixel points constitute the stitching area. It should be noted that the above method for determining the stitching pixel points is only an example, and in actual applications, it can be flexibly set as needed to determine the stitching area. It can be understood that the more stitching pixel points are determined, the larger the range of the stitching area will be, and the effect of the target image obtained by subsequent processing will theoretically be better.

[0105] Furthermore, the stitching area can be subjected to image smoothing processing to obtain the target image. Exemplarily, the image smoothing processing can be achieved by image automatic compensation technology. Smoothing processing can include eliminating aliasing in the stitching area, wherein the aliasing can be understood as more prominent and uncoordinated pixels in the stitching area. In other words, smoothing processing can be a processing of smooth transition of the color of each pixel in the stitching area, for example, smoothing processing can be achieved by re-determining the color of each pixel in the stitching area. Specifically, a pixel in the stitching area can be obtained, and the color average of each adjacent pixel around the pixel is used as the new color of the pixel. Each pixel in the stitching area is traversed in turn, thereby obtaining each pixel with a new color, which constitutes the stitching area after smoothing processing, thereby making the target image more integrated and unified.

[0106] Step S270: Display the target image in the mixed reality device.

[0107] Among them, step S270 has been described in detail in the above embodiment and will not be repeated here.

[0108] The blind spot display method, device, electronic device, and readable storage medium provided in the embodiments of the present application are based on a mixed reality device worn by a target person in a target vehicle, capturing a first image; capturing a second image based on an image capture device installed in the target vehicle; determining deformation parameters based on the eye posture information, the position information of the image capture device, and obstructing objects in the target vehicle; adjusting the second image based on the deformation parameters to obtain a corrected image; shielding the partial image corresponding to the blind spot in the first image to obtain an alternative image; superimposing the corrected image on the partial image corresponding to the shielded blind spot in the alternative image, and splicing it with the alternative image to obtain the target image; and displaying the target image on the mixed reality device. In the embodiments provided in the present application, precisely because the size of the corrected image is adapted to the size of the partial image corresponding to the blind spot in the first image, the subsequent display effect of the acquired target image can be improved and the sense of fragmentation of the target image can be reduced. In addition, in the embodiments of the present application, the first specified parameter of the alternative image and the second specified parameter of the corrected image can be adjusted through image automatic compensation technology. The adjusted first specified parameter is consistent with the adjusted second specified parameter. This ensures that the target image subsequently obtained based on the corrected image and the alternative image will not appear fragmented, thereby improving the display quality of the target image. Furthermore, in the embodiments provided herein, a splicing region can be determined within the intermediate image, and then image smoothing processing can be performed on the spliced ​​region to obtain the target image. This avoids problems such as unnatural transitions in the spliced ​​region of the image.

[0109] See also Figure 10 , Figure 10 A method flow chart of a blind area display method provided by an embodiment of the present application is shown. The blind area display method can be applied to Figure 1 In the blind spot display scenario shown in , the method may specifically include steps S310 to S380.

[0110] Step S310: Based on the mixed reality device worn by the target person riding in the target vehicle, a first image is captured, wherein the first image is an image corresponding to the eye posture information of the target person.

[0111] Step S320: Based on the image acquisition device provided on the target vehicle, a second image is acquired, wherein the second image is a real scene image corresponding to the blind spot of the target person's field of vision in the surrounding environment of the target vehicle.

[0112] Step S330: Determine deformation parameters based on the eye posture information, the position information of the image acquisition device, and the blocking object in the target vehicle, where the blocking object is an object that blocks the field of view of the target person and forms the blind spot.

[0113] Step S340: adjusting the second image based on the deformation parameter to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind spot in the field of view in the first image.

[0114] Step S350: synthesizing the first image and the corrected image to replace the local image corresponding to the blind spot in the first image with the local image corresponding to the blind spot in the corrected image to obtain a target image.

[0115] Among them, steps S310 to S350 have been described in detail in the above embodiments and will not be repeated here.

[0116] Step S360: Identify the target image.

[0117] Step S370: When a target object is identified in the target image, prompt information is generated and provided to the target person.

[0118] Step S380: Display the target image in the mixed reality device.

[0119] In some embodiments, after acquiring a target image, the target image may be identified. Recognizing the target image may include identifying an object within the target image. When the target object is identified within the target image, prompt information is generated and provided to the target person. For example, the target image can be identified and the prompt information generated in conjunction with an intelligent driving system operating in the target vehicle.

[0120] The target objects may be pre-set objects or specified by the target person. For example, the target person may specify objects of certain types as target objects through the intelligent driving system. For example, the target objects may include pedestrians, roadblocks, and signboards.

[0121] Recognizing objects in a target image can also be achieved using a model trained on a neural network. Specifically, the target image can be used as the input of the model to generate the model's output. For example, the output can be text information, which can include text that specifically identifies the object in the target image. The text information can also include information determining whether the target object is present in the target image, such as a "yes" or "no" answer.

[0122] Furthermore, if a target object is identified in the target image, prompt information is generated and provided to the target person. If a target object is identified in the target image, it can indicate that the target vehicle may be at risk of an accident, and prompt information can be generated and provided to the target person.

[0123] In some embodiments, the prompt information may be a voice prompt, such as a voice prompt corresponding to "Please slow down." After the prompt information is generated, it may be played through the target vehicle's onboard system or directly through a speaker configured in a mixed reality device worn by the target person.

[0124] In some other embodiments, the prompt information may also be a text prompt, such as a text prompt corresponding to "Please slow down." After the prompt information is generated, the prompt information may be displayed on a mixed reality device worn by the target person.

[0125] For some other implementation methods, the prompt information may also include voice prompts and text prompts.

[0126] Therefore, when the target person includes a driver, the driver can be reminded to pay attention to safety in a timely manner, reducing the probability of an accident.

[0127] It is understood that after acquiring the target image, the target image can be displayed in the mixed reality device. The detailed method of displaying the target image can be found in the above embodiment and will not be repeated here.

[0128] It should be noted that the above embodiment, in which after acquiring a target image, the target image is first recognized and then displayed on the mixed reality device, is merely an example. In actual applications, the order of performing target image recognition and displaying the target image on the mixed reality device is not specifically limited and can be performed simultaneously or sequentially.

[0129] Step S390: Acquire the current status information of the target vehicle, where the status information includes at least one of vehicle speed information, tire pressure information, and navigation information.

[0130] Step S3100: Display the status information in the mixed reality device.

[0131] Optionally, the mixed reality device can also serve as an extension of the information display to display status information for the target person. Therefore, in some embodiments, the current status information of the target vehicle can also be obtained and then displayed on the mixed reality device. The status information may include at least one of vehicle speed information, tire pressure information, and navigation information. Exemplarily, the intelligent driving system running in the target vehicle can capture data on the current status of the target vehicle and generate status information. This can improve the convenience of the target person in viewing the status information.

[0132] For some embodiments, displaying status information in a mixed reality device may involve displaying the status information in a status display area on a target image. Exemplarily, the status information may be displayed in the form of text information, thereby overlaying the status display area of ​​the target image with the text information, thereby displaying the target image including the status information in the mixed reality device. In another exemplary embodiment, the status information may also be displayed in the form of text information combined with graphic information, thereby overlaying the status display area of ​​the target image with text information and graphic information, thereby displaying the target image including the status information in the mixed reality device.

[0133] Optionally, the status display area may be a pre-set area in the target image, for example, the area in the right corner of the target image may be used as the status display area; for another example, the area in the upper left corner of the target image may be used as the status display area.

[0134] Optionally, the status display area may also be an area selected by the target person in the target image, so that the selected area may be used as the status display area to display status information.

[0135] The blind area display method, device, electronic equipment and readable storage medium provided by the embodiments of the present application collect a first image based on a mixed reality device worn by a target person riding a target vehicle; collect a second image based on an image collection device arranged on the target vehicle; determine a deformation parameter based on the eye pose information, the position information of the image collection device and the occlusion object in the target vehicle; adjust the second image based on the deformation parameter to obtain a corrected image; synthesize the first image and the corrected image to replace the local image corresponding to the blind area of the field of view in the first image with the local image corresponding to the blind area of the field of view in the corrected image to obtain a target image; identify the target image; generate a prompt information and provide the prompt information to the target person in the case that the target object is identified in the target image; display the target image in the mixed reality device; and display the current state information of the target vehicle in the mixed reality device. In the embodiments provided by the present application, the driver can be reminded of safety in time through the prompt information, and the probability of accidents is reduced. Moreover, the current state information of the target vehicle can be realized through the mixed reality device, and the convenience of the target person in checking the state information is improved.

[0136] Please refer to Figure 11 , Figure 11 A structure block diagram of a blind area display device 1100 provided by the embodiments of the present application is shown, which includes a first collection unit 1110, a second collection unit 1120, a deformation parameter determination unit 1130, an adjustment unit 1140, a processing unit 1150 and a display unit 1160.

[0137] The first collection unit 1110 is configured to collect a first image based on a mixed reality device worn by a target person riding a target vehicle, wherein the first image is an image corresponding to the eye pose information of the target person.

[0138] The second collection unit 1120 is configured to collect a second image based on an image collection device arranged on the target vehicle, wherein the second image is a real scene image corresponding to the blind area of the field of view of the target person in the environment around the target vehicle.

[0139] The deformation parameter determination unit 1130 is configured to determine a deformation parameter based on the eye pose information, the position information of the image collection device and the occlusion object in the target vehicle, wherein the occlusion object is an object that occludes the field of view of the target person to form the blind area of the field of view.

[0140] The adjusting unit 1140 is configured to adjust the second image based on the deformation parameter to obtain a modified image, wherein a size of the modified image is adapted to a size of the local image corresponding to the blind area in the first image.

[0141] The processing unit 1150 is configured to synthesize the first image and the modified image to replace the local image corresponding to the blind area in the first image with the local image corresponding to the blind area in the modified image to obtain a target image.

[0142] Optionally, the processing unit 1150 is further configured to shield the local image corresponding to the blind area in the first image to obtain a candidate image; and superimpose the modified image to the local image corresponding to the blind area in the candidate image to splice the candidate image to obtain the target image.

[0143] Optionally, the processing unit 1150 is further configured to adjust a first designated parameter of the candidate image; and adjust a second designated parameter of the modified image, wherein the first designated parameter and the second designated parameter each include at least one of exposure, sharpness, chromatic aberration and color tone, the adjusted first designated parameter is consistent with the adjusted second designated parameter; superimpose the adjusted modified image to the local image corresponding to the blind area in the adjusted candidate image to splice the adjusted candidate image to obtain the target image.

[0144] The display unit 1160 is configured to display the target image in the mixed reality device.

[0145] Optionally, the display unit 1160 is further configured to identify the target image; in a case where a target object is identified in the target image, generate prompt information and provide the prompt information to a target person; and display the target image in the mixed reality device.

[0146] Optionally, the display unit 1160 is further configured to obtain current state information of the target vehicle, the state information including at least one of vehicle speed information, tire pressure information and navigation information; and display the state information in the mixed reality device.

[0147] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0148] In the several embodiments provided in this application, the coupling between the units can be electrical, mechanical, or other forms of coupling. In addition, the functional units in the various embodiments of this application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or software functional units.

[0149] See also Figure 12 , Figure 12 The following is a block diagram of an electronic device 1200 provided in an embodiment of the present application. The electronic device 1200 may be a vehicle-mounted system, which may be installed in a vehicle. The electronic device 1200 in the present application may include one or more of the following components: a processor 1211, a memory 1212, and one or more application programs. The processor 1211 is electrically connected to the memory 1212, and the one or more application programs are configured to execute the methods described in the aforementioned embodiments of the blind spot display method.

[0150] The processor 1211 may include one or more processing cores. The processor 1211 utilizes various interfaces and circuits to connect various components within the electronic device 1200. It executes various functions and processes data for the electronic device 1200 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1212, as well as accessing data stored in the memory 1212. Optionally, the processor 1211 may be implemented in the form of at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 1211 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and computer programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1211, but may be implemented by a separate communication chip. Specifically, the method described in the above embodiment may be executed by one or more processors 1211.

[0151] For some embodiments, the memory 1212 may include a random access memory (RAM) or a read-only memory (ROM). The memory 1212 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1212 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, and the like. The data storage area may also store data created by the electronic device 1200 during use.

[0152] See also Figure 13 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 1300 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0153] Computer-readable storage medium 1300 can be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Alternatively, computer-readable storage medium 1300 includes a non-transitory computer-readable storage medium. Computer-readable storage medium 1300 has storage space for program code 1310 that executes any method step of the above method. These program codes can be read from or written into one or more computer program products. Program code 1310 can be compressed, for example, in an appropriate form.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A blind area display method, characterized in that: include: Capturing a first image based on a mixed reality device worn by a target person riding in a target vehicle, wherein the first image is an image corresponding to eye posture information of the target person; Capturing a second image based on an image acquisition device provided on the target vehicle, wherein the second image is a real-scene image corresponding to a blind spot of the target person in the surrounding environment of the target vehicle; Determining a deformation parameter based on the eye posture information, the position information of the image acquisition device, and an obstructing object in the target vehicle, wherein the obstructing object is an object that blocks the field of view of the target person and forms the blind spot; Adjusting the second image based on the deformation parameter to obtain a corrected image, wherein the size of the corrected image is adapted to the size of the local image corresponding to the blind area of ​​the visual field in the first image; synthesizing the first image and the corrected image to replace the partial image corresponding to the blind area of ​​the visual field in the first image with the partial image corresponding to the blind area of ​​the visual field in the corrected image, thereby obtaining a target image; The target image is displayed in the mixed reality device.

2. The method according to claim 1, characterized in that The synthesizing the first image and the corrected image to replace the partial image corresponding to the blind area of ​​the visual field in the first image with the partial image corresponding to the blind area of ​​the visual field in the corrected image to obtain a target image includes: shielding a partial image corresponding to a blind area of ​​the visual field in the first image to obtain an alternative image; The corrected image is superimposed on a local image corresponding to the shielded blind area of ​​the visual field in the candidate image, and is spliced ​​with the candidate image to obtain the target image.

3. The method according to claim 2, characterized in that The step of superimposing the corrected image onto a partial image corresponding to a blocked blind area of ​​the visual field in the candidate image and splicing the partial image with the candidate image to obtain the target image includes: adjusting at least one of a first specified parameter of the candidate image and a second specified parameter of the corrected image so that the first specified parameter is consistent with the second specified parameter, wherein the first specified parameter and the second specified parameter both include at least one of exposure, sharpness, color difference, and hue; After at least one of the first specified parameter and the second specified parameter is adjusted, the corrected image is superimposed on the local image corresponding to the blocked blind area of ​​the visual field in the alternative image, and is spliced ​​with the alternative image to obtain the target image.

4. The method according to claim 2, characterized in that The step of superimposing the corrected image onto a partial image corresponding to a blocked blind area of ​​the visual field in the candidate image and splicing the partial image with the candidate image to obtain the target image includes: Superimposing the corrected image on a partial image corresponding to the blocked blind area of ​​the visual field in the candidate image, and splicing it with the candidate image to obtain an intermediate image; determining a stitching region in the intermediate image; Performing image smoothing processing on the stitching area to obtain the target image.

5. The method according to claim 1, wherein Displaying the target image in the mixed reality device includes: Recognizing the target image; When a target object is identified in the target image, generating prompt information and providing the prompt information to the target person; The target image is displayed in the mixed reality device.

6. The method according to claim 1, characterized in that After displaying the target image in the mixed reality device, the method further includes: Acquiring current status information of the target vehicle, the status information including at least one of vehicle speed information, tire pressure information, and navigation information; The status information is displayed in the mixed reality device.

7. A blind spot display device, characterized in that: include: A first acquisition unit is configured to acquire a first image based on a mixed reality device worn by a target person riding in a target vehicle, wherein the first image is an image corresponding to eye posture information of the target person; A second acquisition unit is configured to acquire a second image based on an image acquisition device provided on the target vehicle, wherein the second image is a real scene image corresponding to a blind spot of the target person in the surrounding environment of the target vehicle; a deformation parameter determining unit, configured to determine a deformation parameter based on the eye posture information, the position information of the image acquisition device, and an obstructing object in the target vehicle, wherein the obstructing object is an object that obstructs the field of view of the target person and forms the blind spot; an adjusting unit, configured to adjust the second image based on the deformation parameter to obtain a corrected image, wherein a size of the corrected image is adapted to a size of a partial image corresponding to a blind spot in the first image; a processing unit configured to synthesize the first image and the corrected image, so as to replace the partial image corresponding to the blind area of ​​the visual field in the first image with the partial image corresponding to the blind area of ​​the visual field in the corrected image, thereby obtaining a target image; A display unit is configured to display the target image in the mixed reality device.

8. An electronic device, characterized in that: include: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images