Panoramic image construction method and device, equipment and storage medium

Abstract

The invention relates to the field of image processing, and discloses a panoramic image construction method, device and equipment and a storage medium, and the method comprises the steps: obtaining a camera image corresponding to each camera direction based on each vehicle camera; determining a blind area image corresponding to each preset blind area according to each preset blind area corresponding to the vehicle, the advancing direction and the steering direction of the vehicle and the historical video stream corresponding to each vehicle camera; and performing image splicing on the camera images and the blind area images to obtain a current panoramic image. According to the technical scheme, the blind area part in the panoramic image is filled with the historical images of the cameras, and the accuracy of the panoramic image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular, to a panoramic image construction method, apparatus, device, and storage medium. Background Art

[0002] Images around the vehicle are captured by four cameras installed on the vehicle, namely, a front camera, a rear camera, a left camera, and a right camera. Then, a panoramic image can be synthesized. This panoramic image is like a panoramic view of the vehicle taken from above. Then, this panoramic image can be further displayed on an in-vehicle screen to provide a basis for vehicle control.

[0003] However, due to the installation positions of the cameras on the vehicle and the limitations of the field of view of the cameras themselves, each camera has a blind spot for shooting. In this case, there will also be a blind spot for synthesizing the panoramic image, which will affect subsequent vehicle control judgments and pose a certain safety hazard.

[0004] In view of this, the present invention is specifically proposed. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a panoramic image construction method, apparatus, device, and storage medium, which realizes filling the blind spot part in the panoramic image with the historical images of each camera, and improves the accuracy of the panoramic image.

[0006] An embodiment of the present invention provides a panoramic image construction method, which includes:

[0007] Based on each vehicle camera, obtain camera images corresponding to the directions of each camera;

[0008] According to each preset blind spot area corresponding to the vehicle, the traveling direction of the vehicle, the steering direction, and the historical video streams corresponding to each vehicle camera, determine blind spot images corresponding to each preset blind spot area;

[0009] Perform image stitching on each camera image and each blind spot image to obtain a current panoramic image.

[0010] An embodiment of the present invention provides a panoramic image construction apparatus, which includes:

[0011] A camera image acquisition module, configured to obtain camera images corresponding to the directions of each camera based on each vehicle camera;

[0012] A blind spot image determination module, configured to determine blind spot images corresponding to each preset blind spot area according to each preset blind spot area corresponding to the vehicle, the traveling direction of the vehicle, and the historical video streams corresponding to each vehicle camera;

[0013] An image stitching module, configured to stitch each camera image and each blind area image to obtain a current panoramic image.

[0014] An embodiment of the present invention provides an electronic device, which includes:

[0015] A processor and a memory;

[0016] The processor is configured to execute the steps of the panoramic image construction method described in any of the embodiments by calling a program or instruction stored in the memory.

[0017] An embodiment of the present invention provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction causes a computer to execute the steps of the panoramic image construction method described in any of the embodiments.

[0018] The embodiments of the present invention have the following technical effects:

[0019] Based on each vehicle camera, camera images corresponding to the directions of each camera are obtained to determine real partial images. Furthermore, according to each preset blind area region corresponding to the vehicle, the traveling direction of the vehicle, the steering direction, and the historical video streams corresponding to each vehicle camera, blind area images corresponding to each preset blind area region are determined, so as to use the images in the historical video streams as blind area images for subsequent filling. Each camera image and each blind area image are stitched to obtain a current panoramic image, achieving the effect of filling the blind area part in the panoramic image with the historical images of each camera and improving the accuracy of the panoramic image. Description of the Drawings

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of a panoramic image construction method provided by an embodiment of the present invention;

[0022] Figure 2 It is a flowchart of another panoramic image construction method provided by an embodiment of the present invention;

[0023] Figure 3 It is a schematic diagram of each camera image, each preset blind area region, and each video stream sampling region provided by an embodiment of the present invention;

[0024] Figure 4It is a schematic structural diagram of a panoramic image construction device provided by an embodiment of the present invention;

[0025] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments of the present invention fall within the scope protected by the present invention.

[0027] The panoramic image construction method provided by the embodiment of the present invention is mainly applicable to the situation of constructing a panoramic image corresponding to a vehicle and improving the accuracy of the blind area in the panoramic image. The panoramic image construction method provided by the embodiment of the present invention can be executed by a panoramic image construction device or an electronic device.

[0028] Figure 1 It is a flowchart of a panoramic image construction method provided by an embodiment of the present invention. Refer to Figure 1 and the panoramic image construction method specifically includes:

[0029] S110. Obtain camera images corresponding to the directions of respective vehicle cameras based on the respective vehicle cameras.

[0030] Among them, a vehicle camera is a camera installed on a vehicle for capturing surrounding images, including a front camera, a rear camera, a left camera, and a right camera. The camera direction refers to the direction in which the camera captures images, which may include front, rear, left, and right. A camera image is an image directly captured by each vehicle camera, and these images cannot be directly stitched together and there are blind areas in the capture.

[0031] Specifically, each vehicle camera installed on the vehicle captures the surroundings of the vehicle in real time, that is, images in the directions of respective cameras can be captured, and thus camera images corresponding to each vehicle camera can be obtained.

[0032] S120. Determine blind area images corresponding to respective preset blind areas according to the respective preset blind areas corresponding to the vehicle, the traveling direction of the vehicle, the steering direction, and the historical video streams corresponding to the respective vehicle cameras.

[0033] Among them, the preset blind area is the area that cannot be covered by real images when each vehicle camera takes pictures. The traveling direction refers to the straight-ahead direction of the vehicle, including forward and backward, that is, advancing and retreating. The steering direction is the direction of vehicle direction change, including left and right, that is, turning left and turning right. The historical video stream is the video stream captured by the vehicle camera over a period of time in the past. The blind area image is the image used to fill the preset blind area.

[0034] Specifically, according to the traveling direction and steering direction of the vehicle, it can be determined which vehicle camera has captured each preset blind area of the vehicle in the past period of time. The images of each preset blind area captured previously can be utilized, that is, the historical video streams corresponding to these vehicle cameras are obtained. Select the image frames with appropriate frames, and extract the images of each preset blind area from them as the blind area images corresponding to each preset blind area.

[0035] S130. Perform image stitching on each camera image and each blind area image to obtain the current panoramic image.

[0036] Among them, the current panoramic image is the panoramic image of the vehicle obtained through image stitching, which looks like an overhead view of the vehicle scene image.

[0037] Specifically, determine the stitching positions of each camera image and each blind area image, arrange each camera image and each blind area image according to the stitching positions, and perform image stitching operations on these camera images and blind area images to obtain the current panoramic image.

[0038] The present invention has the following technical effects: Based on each vehicle camera, the camera images corresponding to each camera direction are obtained to determine the real partial images. Furthermore, according to each preset blind area of the vehicle, the traveling direction of the vehicle, the steering direction, and the historical video streams corresponding to each vehicle camera, the blind area images corresponding to each preset blind area are determined, so as to use the images in the historical video stream as the blind area images for subsequent filling. Perform image stitching on each camera image and each blind area image to obtain the current panoramic image, achieving the effect of filling the blind area part in the panoramic image with the historical images of each camera and improving the accuracy of the panoramic image.

[0039] Figure 2 It is the flowchart of another panoramic image construction method provided by the embodiment of the present invention. Refer to Figure 2 This panoramic image construction method specifically includes:

[0040] S210. Based on each vehicle camera, obtain the camera images corresponding to each camera direction.

[0041] S220. According to the traveling direction and steering direction of the vehicle, determine the target cameras corresponding to each preset blind area from each vehicle camera.

[0042] Among them, the target camera is a vehicle camera that has captured images of the preset blind spot area within a past period of time. It can be understood that each preset blind spot area can correspond to a target camera, and the target cameras corresponding to different preset blind spot areas can be the same or different.

[0043] Specifically, in combination with the traveling direction and the steering direction of the vehicle, it can be deduced which vehicle cameras have captured each preset blind spot area within a past period of time. Therefore, for each preset blind spot area, the vehicle camera that has captured the video stream covering this area within a past period of time can be used as the target camera corresponding to this preset blind spot area.

[0044] Based on the above example, the preset blind spot area includes the four-corner blind spot area and the traveling blind spot area. The traveling blind spot area is the front blind spot area when the traveling direction is forward, and the rear blind spot area when the traveling direction is right. The four-corner blind spot area includes the upper left blind spot area, the lower left blind spot area, the upper right blind spot area, and the lower right blind spot area, as Figure 3 shown.

[0045] Based on the above example, the following method can be used to determine the target camera corresponding to each preset blind spot area from each vehicle camera according to the traveling direction and the steering direction of the vehicle:

[0046] In response to the traveling direction being forward, determine the front camera as the target camera corresponding to the front blind spot area, and determine the target camera corresponding to each four-corner blind spot area according to the steering direction;

[0047] In response to the traveling direction being backward, determine the rear camera as the target camera corresponding to the rear blind spot area, and determine the target camera corresponding to each four-corner blind spot area according to the steering direction.

[0048] Specifically, if the vehicle's traveling direction is forward, it can be known that the target camera corresponding to the front blind area needs to be determined, and there is no need to determine the target camera corresponding to the rear blind area, because when traveling forward, there is no danger in the rear blind area. When traveling forward, it can be deduced that the front blind area was captured by the front camera in the past period of time. Therefore, the front camera can be used as the target camera corresponding to the front blind area. The target camera corresponding to the four-corner blind area is related to the steering direction. The vehicle's steering dynamics determine which vehicle camera captured the four-corner blind area in the past period of time. Therefore, in combination with the steering direction, the target camera corresponding to each four-corner blind area is deduced. If the vehicle's traveling direction is backward, it can be known that the target camera corresponding to the rear blind area needs to be determined, and there is no need to determine the target camera corresponding to the front blind area, because when reversing backward, there is no danger in the front blind area. The method for determining the target camera corresponding to the four-corner blind area when reversing backward is similar to that when traveling forward, and will not be elaborated here.

[0049] Based on the above example, if the traveling direction is forward, the following method can be used to determine the target camera corresponding to each four-corner blind area according to the steering direction:

[0050] In response to the steering direction being a left turn, determine that the left camera is the target camera corresponding to the upper left blind area, determine that the rear camera is the target camera corresponding to the lower left blind area, determine that the front camera is the target camera corresponding to the upper right blind area, and determine that the right camera is the target camera corresponding to the lower right blind area;

[0051] In response to the steering direction being a right turn, determine that the front camera is the target camera corresponding to the upper left blind area, determine that the left camera is the target camera corresponding to the lower left blind area, determine that the right camera is the target camera corresponding to the upper right blind area, and determine that the rear camera is the target camera corresponding to the lower right blind area;

[0052] In response to the steering direction being no turn, determine that the front camera is the target camera corresponding to the upper left blind area, determine that the left camera is the target camera corresponding to the lower left blind area, determine that the front camera is the target camera corresponding to the upper right blind area, and determine that the right camera is the target camera corresponding to the lower right blind area.

[0053] Specifically, if the steering direction is to the left, it can be known that each quadrant blind spot area has been captured by the vehicle camera corresponding to the adjacent area on the right (clockwise) of each quadrant blind spot area in the past period of time. Thus, it can be known that the left camera is the target camera corresponding to the upper left blind spot area, the rear camera is the target camera corresponding to the lower left blind spot area, the front camera is the target camera corresponding to the upper right blind spot area, and the right camera is the target camera corresponding to the lower right blind spot area. If the steering direction is to the right, it can be known that each quadrant blind spot area has been captured by the vehicle camera corresponding to the adjacent area on the left (counterclockwise) of each quadrant blind spot area in the past period of time. Thus, it can be known that the front camera is the target camera corresponding to the upper left blind spot area, the left camera is the target camera corresponding to the lower left blind spot area, the right camera is the target camera corresponding to the upper right blind spot area, and the rear camera is the target camera corresponding to the lower right blind spot area. If the steering direction is not turning, it can be known that the vehicle is in a state of going straight forward without changing direction. The target cameras corresponding to the upper left blind spot area and the upper right blind spot area, which are the quadrant blind spot areas in the front, are both the front camera, and the target cameras corresponding to the quadrant blind spot areas in the rear are the vehicle cameras on the same side. That is, the target camera corresponding to the lower right blind spot area is the right camera, and the target camera corresponding to the lower left blind spot area is the left camera.

[0054] Based on the above example, if the traveling direction is backward, the target cameras corresponding to each quadrant blind spot area can be determined according to the steering direction in the following way:

[0055] In response to the steering direction being to the left, determine that the front camera is the target camera corresponding to the upper left blind spot area, determine that the left camera is the target camera corresponding to the lower left blind spot area, determine that the right camera is the target camera corresponding to the upper right blind spot area, and determine that the rear camera is the target camera corresponding to the lower right blind spot area;

[0056] In response to the steering direction being to the right, determine that the left camera is the target camera corresponding to the upper left blind spot area, determine that the rear camera is the target camera corresponding to the lower left blind spot area, determine that the front camera is the target camera corresponding to the upper right blind spot area, and determine that the right camera is the target camera corresponding to the lower right blind spot area;

[0057] In response to the steering direction being not turning, determine that the left camera is the target camera corresponding to the upper left blind spot area, determine that the rear camera is the target camera corresponding to the lower left blind spot area, determine that the right camera is the target camera corresponding to the upper right blind spot area, and determine that the rear camera is the target camera corresponding to the lower right blind spot area.

[0058] Specifically, if the steering direction is to the left, it can be known that each four-corner blind area has been captured by the vehicle camera corresponding to the left (counterclockwise) adjacent area of each four-corner blind area in the past period of time. Thus, it can be known that the front camera is the target camera corresponding to the upper left blind area, the left camera is the target camera corresponding to the lower left blind area, the right camera is the target camera corresponding to the upper right blind area, and the rear camera is the target camera corresponding to the lower right blind area. If the steering direction is to the right, it can be known that each four-corner blind area has been captured by the vehicle camera corresponding to the left (clockwise) adjacent area of each four-corner blind area in the past period of time. Thus, it can be known that the left camera is the target camera corresponding to the upper left blind area, the rear camera is the target camera corresponding to the lower left blind area, the front camera is the target camera corresponding to the upper right blind area, and the right camera is the target camera corresponding to the lower right blind area. If the steering direction is not turning, it can be known that the vehicle is in a state of reversing backward without direction change. The target cameras corresponding to the lower left blind area and the lower right blind area, which are the four-corner blind areas at the rear, are both the rear camera, and the target cameras corresponding to the four-corner blind areas in the front are the vehicle cameras on the same side. That is, the target camera corresponding to the upper right blind area is the right camera, and the target camera corresponding to the upper left blind area is the left camera.

[0059] S230. For each preset blind area, obtain the historical video stream collected by the target camera corresponding to the preset blind area, and determine the blind area image corresponding to the preset blind area according to the preset time interval, the historical video stream, and the preset blind area.

[0060] Among them, the preset time interval is a preset value required to find the image frame at the appropriate timestamp position in the historical video stream, such as 200 ms, etc.

[0061] Specifically, for each preset blind area, retrieve the historical video stream collected by the target camera corresponding to the preset blind area, determine the image frame for filling in the historical video stream according to the preset time interval. Furthermore, match the camera images adjacent to the preset blind area in the image frame, and determine the partial image used to fill the preset blind area as the blind area image corresponding to the preset blind area.

[0062] Based on the above example, the blind area image corresponding to the preset blind area can be determined according to the preset time interval, the historical video stream, and the preset blind area in the following way:

[0063] Determine the historical image frame corresponding to the preset time interval from the historical video stream;

[0064] According to the area category of the preset blind area, determine the video stream sampling area corresponding to the preset blind area;

[0065] Determine the blind - area image corresponding to the preset blind - area region from the historical image frames according to the video - stream sampling region.

[0066] Among them, the historical image frame is a video image frame in the historical video stream that has a preset time interval with the current moment. The region categories include the four - corner blind - area region and the front blind - area region. The video - stream sampling region is a pre - set region shape, position, and size corresponding to the preset blind - area region.

[0067] Specifically, according to the preset time interval, determine the video image frame located at the preset time - interval position before the current moment in the historical video stream as the historical image frame. According to the region category of the preset blind - area region, the shape and size of the video - stream sampling region can be determined. According to the traveling direction or turning direction, the position in front of, behind, on the left, or on the right of the preset blind - area region can be determined as the position of the video - stream sampling region.

[0068] Based on the above example, the video - stream sampling region corresponding to the preset blind - area region can be determined in the following way according to the region category of the preset blind - area region:

[0069] In response to the region category of the preset blind - area region being the traveling blind - area region, determine the video - stream sampling region corresponding to the preset blind - area region according to the traveling direction and the size of the preset blind - area region;

[0070] In response to the region category of the preset blind - area region being the four - corner blind - area region, determine the video - stream sampling region corresponding to the preset blind - area region according to the turning direction and the size of the preset blind - area region.

[0071] Specifically, if the region category of the preset blind - area region is the traveling blind - area region, when the traveling direction is forward, it is determined as the front blind - area region; when the traveling direction is backward, it is determined as the rear blind - area region. Its shape is a rectangle, and the length and width are preset values. Accordingly, the shape, size, and position of the video - stream sampling region can be determined to correspond to the preset blind - area region, facilitating subsequent image stitching. If the region category of the preset blind - area region is the four - corner blind - area region, first determine that the shape of the preset blind - area region is a triangle, and its size is determined by the preset base and height. And, combined with the turning direction, the position of the video - stream sampling region corresponding to each preset blind - area region can be determined, that is, the position of the partial image to be filled in the historical image frame, facilitating subsequent image stitching. The forward - traveling video - stream sampling region, reverse - traveling video - stream sampling region, left - turning video - stream sampling region, and right - turning video - stream sampling region are all as Figure 3 shown.

[0072] It can be understood that the specific preset time interval, the sizes and positions corresponding to each preset blind - area region can be pre - calibrated according to vehicle types and camera types, etc.

[0073] S240. Stitch the images of each camera with the images of each blind spot area to obtain the current panoramic image.

[0074] The present invention has the following technical effects: By determining the target cameras corresponding to each preset blind spot area from each vehicle camera according to the traveling direction and steering direction of the vehicle, for each preset blind spot area, obtaining the historical video stream collected by the target camera corresponding to the preset blind spot area, and determining the blind spot image corresponding to the preset blind spot area according to the preset time interval, the historical video stream, and the preset blind spot area, it realizes determining the target camera by combining the traveling direction and the steering direction, and determining the corresponding blind spot image from the historical video stream collected by the target camera, improving the accuracy of determining the blind spot image.

[0075] Figure 4 It is a schematic structural diagram of a panoramic image construction device provided by an embodiment of the present invention. As Figure 4 shown, the device includes: a camera image acquisition module 310, a blind spot image determination module 320, and an image stitching module 330.

[0076] Among them, the camera image acquisition module 310 is used to acquire the camera images corresponding to the directions of each vehicle camera based on each vehicle camera; the blind spot image determination module 320 is used to determine the blind spot images corresponding to each preset blind spot area according to each preset blind spot area corresponding to the vehicle, the traveling direction of the vehicle, the steering direction, and the historical video streams corresponding to each vehicle camera; the image stitching module 330 is used to stitch the images of each camera with the images of each blind spot area to obtain the current panoramic image.

[0077] Based on the above example, optionally, the blind spot image determination module 320 is further used to determine the target cameras corresponding to each preset blind spot area from each vehicle camera according to the traveling direction and steering direction of the vehicle; for each preset blind spot area, obtain the historical video stream collected by the target camera corresponding to the preset blind spot area, and determine the blind spot image corresponding to the preset blind spot area according to the preset time interval, the historical video stream, and the preset blind spot area.

[0078] Based on the above example, optionally, the preset blind area region includes a four-corner blind area region and a traveling blind area region. The traveling blind area region is a front blind area region when the traveling direction is forward, and a rear blind area region when the traveling direction is right. The four-corner blind area region includes an upper left blind area region, a lower left blind area region, an upper right blind area region, and a lower right blind area region. The blind area image determination module 320 is further configured to, in response to the traveling direction being forward, determine the front camera as the target camera corresponding to the front blind area region, and determine the target cameras corresponding to the four-corner blind area regions according to the steering direction; in response to the traveling direction being backward, determine the rear camera as the target camera corresponding to the rear blind area region, and determine the target cameras corresponding to the four-corner blind area regions according to the steering direction.

[0079] Based on the above example, optionally, when the traveling direction is forward, the blind area image determination module 320 is further configured to, in response to the steering direction being left, determine the left camera as the target camera corresponding to the upper left blind area region, determine the rear camera as the target camera corresponding to the lower left blind area region, determine the front camera as the target camera corresponding to the upper right blind area region, and determine the right camera as the target camera corresponding to the lower right blind area region; in response to the steering direction being right, determine the front camera as the target camera corresponding to the upper left blind area region, determine the left camera as the target camera corresponding to the lower left blind area region, determine the right camera as the target camera corresponding to the upper right blind area region, and determine the rear camera as the target camera corresponding to the lower right blind area region; in response to the steering direction being non-steering, determine the front camera as the target camera corresponding to the upper left blind area region, determine the left camera as the target camera corresponding to the lower left blind area region, determine the front camera as the target camera corresponding to the upper right blind area region, and determine the right camera as the target camera corresponding to the lower right blind area region.

[0080] Based on the above example, optionally, when the traveling direction is backward, the blind area image determination module 320 is further configured to, in response to the steering direction being a left turn, determine that the front camera is the target camera corresponding to the upper left blind area, determine that the left camera is the target camera corresponding to the lower left blind area, determine that the right camera is the target camera corresponding to the upper right blind area, and determine that the rear camera is the target camera corresponding to the lower right blind area; in response to the steering direction being a right turn, determine that the left camera is the target camera corresponding to the upper left blind area, determine that the rear camera is the target camera corresponding to the lower left blind area, determine that the front camera is the target camera corresponding to the upper right blind area, and determine that the right camera is the target camera corresponding to the lower right blind area; in response to the steering direction being no turn, determine that the left camera is the target camera corresponding to the upper left blind area, determine that the rear camera is the target camera corresponding to the lower left blind area, determine that the right camera is the target camera corresponding to the upper right blind area, and determine that the rear camera is the target camera corresponding to the lower right blind area.

[0081] Based on the above example, optionally, the blind area image determination module 320 is further configured to determine a historical image frame corresponding to a preset time interval from the historical video stream; determine a video stream sampling area corresponding to the preset blind area according to the area category of the preset blind area; and determine a blind area image corresponding to the preset blind area from the historical image frames according to the video stream sampling area.

[0082] Based on the above example, optionally, when the area category of the preset blind area is a traveling blind area, the blind area image determination module 320 is further configured to determine the video stream sampling area corresponding to the preset blind area according to the traveling direction and the size of the preset blind area; when the area category of the preset blind area is a four-corner blind area, determine the video stream sampling area corresponding to the preset blind area according to the steering direction and the size of the preset blind area.

[0083] For the convenience of description, the above device is described by function as various modules respectively. Of course, when implementing the present application, the functions of each module can be implemented in one or more software and / or hardware.

[0084] The device in the above embodiment is used to implement the corresponding panoramic image construction method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated herein.

[0085] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 5 shown, the electronic device 400 includes one or more processors 401 and a memory 402.

[0086] The processor 401 can be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 400 to perform desired functions.

[0087] The memory 402 can include one or more computer program products, and the computer program products can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions can be stored on the computer-readable storage media, and the processor 401 can run the program instructions to implement the panoramic image construction method of any embodiment of the present invention described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. can also be stored in the computer-readable storage media.

[0088] In one example, the electronic device 400 can further include: an input device 403 and an output device 404, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). The input device 403 can include, for example, a keyboard, a mouse, etc. The output device 404 can output various information to the outside, including early warning prompt information, braking force, etc. The output device 404 can include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0089] Of course, for simplicity, Figure 5 only some of the components related to the present invention in the electronic device 400 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 400 can further include any other appropriate components.

[0090] In addition to the above methods and devices, embodiments of the present invention can also be computer program products, which include computer program instructions, and when the computer program instructions are run by a processor, the processor is caused to execute the steps of the panoramic image construction method provided by any embodiment of the present invention.

[0091] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present invention. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0092] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are run by a processor, the processor is caused to execute the steps of the panoramic image construction method provided by any embodiment of the present invention.

[0093] The computer-readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0094] It should be noted that the terms used in the present invention are only for describing specific embodiments and do not limit the scope of the present application. As shown in the specification of the present invention, unless the context clearly indicates an exception, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. The term "comprising", "including", or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, or device comprising the element.

[0095] It should also be noted that the orientation or positional relationship indicated by terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention. Unless otherwise clearly specified and defined, terms such as "installed", "connected", "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A panoramic image construction method, characterized in that: include: Based on the cameras of each vehicle, obtain the camera image corresponding to the direction of each camera; Determine a blind spot image corresponding to each preset blind spot area according to each preset blind spot area corresponding to the vehicle, the traveling direction and the turning direction of the vehicle, and the historical video stream corresponding to each vehicle camera; The images of each camera and the images of each blind area are stitched together to obtain the current panoramic image.

2. The method according to claim 1, characterized in that The method of determining the blind spot images corresponding to the preset blind spots according to the preset blind spots corresponding to the vehicle, the traveling direction and the turning direction of the vehicle, and the historical video streams corresponding to the cameras of the vehicle includes: According to the traveling direction and the turning direction of the vehicle, determining the target camera corresponding to each preset blind spot area from each vehicle camera; For each preset blind spot area, a historical video stream collected by a target camera corresponding to the preset blind spot area is obtained, and a blind spot image corresponding to the preset blind spot area is determined according to a preset time interval, the historical video stream and the preset blind spot area.

3. The method according to claim 2, characterized in that The preset blind area includes four corner blind areas and a traveling blind area. The traveling blind area is a front blind area when the traveling direction is forward, and is a rear blind area when the traveling direction is right. The four corner blind areas include an upper left blind area, a lower left blind area, an upper right blind area, and a lower right blind area. The step of determining target cameras corresponding to the preset blind spots from the cameras of each vehicle according to the traveling direction and the turning direction of the vehicle includes: In response to the traveling direction being forward, determining that the front camera is the target camera corresponding to the front blind spot area, and determining the target cameras corresponding to the blind spots in each of the four corners according to the turning direction; In response to the traveling direction being backward, the rear camera is determined to be the target camera corresponding to the rear blind spot area, and according to the turning direction, the target cameras corresponding to the blind spots in each of the four corners are determined.

4. The method according to claim 3, characterized in that The traveling direction is forward, and the determining of the target cameras corresponding to the blind spots in each of the four corners according to the turning direction includes: In response to the turning direction being a left turn, the left camera is determined to be the target camera corresponding to the upper left blind spot area, the rear camera is determined to be the target camera corresponding to the lower left blind spot area, the front camera is determined to be the target camera corresponding to the upper right blind spot area, and the right camera is determined to be the target camera corresponding to the lower right blind spot area; In response to the turning direction being a right turn, the front camera is determined to be a target camera corresponding to an upper left blind spot area, the left camera is determined to be a target camera corresponding to a lower left blind spot area, the right camera is determined to be a target camera corresponding to an upper right blind spot area, and the rear camera is determined to be a target camera corresponding to a lower right blind spot area; In response to the turning direction being no turning, the front camera is determined to be the target camera corresponding to the upper left blind spot area, the left camera is determined to be the target camera corresponding to the lower left blind spot area, the front camera is determined to be the target camera corresponding to the upper right blind spot area, and the right camera is determined to be the target camera corresponding to the lower right blind spot area.

5. The method according to claim 3, characterized in that: The traveling direction is backward, and determining the target cameras corresponding to the blind areas at the four corners according to the turning direction includes: In response to the turning direction being a left turn, determining the front camera as a target camera corresponding to an upper left blind spot area, determining the left camera as a target camera corresponding to a lower left blind spot area, determining the right camera as a target camera corresponding to an upper right blind spot area, and determining the rear camera as a target camera corresponding to a lower right blind spot area; In response to the turning direction being a right turn, the left camera is determined to be a target camera corresponding to an upper left blind spot area, the rear camera is determined to be a target camera corresponding to a lower left blind spot area, the front camera is determined to be a target camera corresponding to an upper right blind spot area, and the right camera is determined to be a target camera corresponding to a lower right blind spot area; In response to the turning direction being no turning, the left camera is determined to be the target camera corresponding to the upper left blind spot area, the rear camera is determined to be the target camera corresponding to the lower left blind spot area, the right camera is determined to be the target camera corresponding to the upper right blind spot area, and the rear camera is determined to be the target camera corresponding to the lower right blind spot area.

6. The method according to claim 2, characterized in that The determining, according to the preset time interval, the historical video stream and the preset blind area, a blind area image corresponding to the preset blind area includes: Determining historical image frames corresponding to a preset time interval from the historical video stream; Determining a video stream sampling area corresponding to the preset blind area according to the area category of the preset blind area; According to the video stream sampling area, a blind area image corresponding to the preset blind area is determined from the historical image frames.

7. The method according to claim 6, characterized in that The determining, according to the area category of the preset blind area, a video stream sampling area corresponding to the preset blind area includes: In response to the area category of the preset blind area being a traveling blind area, determining a video stream sampling area corresponding to the preset blind area according to the traveling direction and the size of the preset blind area; In response to the area category of the preset blind area being a four-corner blind area, a video stream sampling area corresponding to the preset blind area is determined according to the turning direction and the size of the preset blind area.

8. A panoramic image construction device, characterized in that: include: A camera image acquisition module, used to acquire camera images corresponding to the directions of each camera based on the cameras of each vehicle; A blind spot image determination module is used to determine the blind spot images corresponding to each preset blind spot area according to each preset blind spot area corresponding to the vehicle, the traveling direction and the turning direction of the vehicle, and the historical video streams corresponding to each vehicle camera; The image stitching module is used to stitch the images of each camera with the images of each blind area to obtain the current panoramic image.

9. An electronic device, characterized in that: The electronic device comprises: Processor and memory; The processor is used to execute the steps of the panoramic image construction method according to any one of claims 1 to 8 by calling the program or instruction stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the panoramic image construction method according to any one of claims 1 to 8.

Images