Image determination method, storage medium, electronic device, program product and vehicle

Abstract

The present disclosure relates to an image determination method, a storage medium, an electronic device, a program product and a vehicle, and relates to the field of vehicle technology. The method comprises: obtaining environmental information of a target area, wherein the target area is a detection area corresponding to at least one of the four corners of a vehicle body. Distribution information of obstacles in the target area is determined based on the environmental information, and the distribution information characterizes the number of obstacles and the position of each obstacle. The position of a stitching seam in the target area is determined based on the number of obstacles and the position of each obstacle. A target panoramic image corresponding to the vehicle is determined based on the position of the stitching seam. The present disclosure dynamically determines the position of a stitching seam based on the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. In the case where there are multiple obstacles in the stitching area, the position of the stitching seam can also avoid the obstacles, thereby avoiding the loss of objects at the stitching seam, and the accuracy of the obtained panoramic image is higher.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular, to an image determination method, a storage medium, an electronic device, a program product, and a vehicle. Background Art

[0002] The vehicle's panoramic surround image is stitched together from images taken by the front, rear, left, and right wide-angle cameras. The four corners of the vehicle are where the stitching seams are located. Since the four images overlap in the stitching area, objects may be lost at the stitching seams. Summary of the invention

[0003] The purpose of the present disclosure is to provide an image determination method, a storage medium, an electronic device, a program product and a vehicle, so as to improve the accuracy of generating a panoramic image of a vehicle.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for determining an image is provided, the method comprising:

[0005] Acquiring environmental information of a target area, wherein the target area is a detection area corresponding to at least one of the four corners of the vehicle body;

[0006] Determine distribution information of obstacles in the target area according to the environmental information, wherein the distribution information represents the number of obstacles and the position of each obstacle;

[0007] Determining the position of the seam of the target area according to the number of the obstacles and the position of each obstacle;

[0008] A target panoramic image corresponding to the vehicle is determined according to the stitching seam position.

[0009] Optionally, the method further comprises:

[0010] Obtaining gear information of the vehicle;

[0011] The target area is determined from a plurality of designated areas according to the gear information, and the designated areas include detection areas corresponding to four corners of a body of the vehicle.

[0012] Optionally, the method further comprises:

[0013] The preset position is used as the stitching seam position of other areas, and the other areas include areas of the multiple designated areas except the target area.

[0014] Optionally, each of the target areas includes a plurality of sub-areas, and each of the sub-areas is covered by detection areas of at least two radar detection devices; and determining the position of the seam of the target area according to the number of obstacles and the position of each obstacle includes:

[0015] When the number of obstacles is zero, a preset position is used as the joint seam position; or,

[0016] In the case that the number of the obstacles is greater than or equal to one, a target sub-region is determined from the multiple sub-regions according to the positions of the obstacles, and a designated position in the target sub-region is used as the joint seam position.

[0017] Optionally, determining a target sub-area from the multiple sub-areas according to the position of the obstacle comprises:

[0018] When the number of the obstacle is one, determining the movement direction of the obstacle according to the position of the obstacle;

[0019] The target sub-region is determined from the sub-regions located in the direction opposite to the moving direction, and the target sub-region is the sub-region farthest from the obstacle.

[0020] Optionally, determining a target sub-area from the multiple sub-areas according to the position of the obstacle includes:

[0021] When the number of the obstacles is greater than one, determining the distribution state of the obstacles according to the positions of the obstacles, wherein the distribution state indicates whether the obstacles are dispersedly distributed or densely distributed;

[0022] The target sub-area is determined according to the distribution state and the position of the obstacle.

[0023] Optionally, determining the target sub-area according to the distribution state and the position of the obstacle includes:

[0024] When the distribution state indicates that the obstacles are dispersedly distributed, determining a target obstacle closest to the vehicle;

[0025] The sub-region farthest from the target obstacle is used as the target sub-region.

[0026] Optionally, determining the target sub-area according to the distribution state and the position of the obstacle includes:

[0027] When the distribution state indicates that the obstacles are densely distributed, determining a target obstacle closest to the vehicle;

[0028] When there are multiple target obstacles closest to the vehicle and the multiple target obstacles are located in different sub-areas, the sub-area with the least number of obstacles is used as the target sub-area.

[0029] Optionally, a plurality of first image acquisition devices for acquiring images of the surrounding environment of the vehicle are arranged on the side of the vehicle, and at least one second image acquisition device for acquiring images of the environment corresponding to the four corners of the vehicle body is arranged respectively at the four corners of the vehicle body; and determining the target panoramic image corresponding to the vehicle according to the position of the stitching seam comprises:

[0030] Acquire a plurality of first environment images by using a plurality of the first image acquisition devices;

[0031] Acquire a plurality of second environment images by using a plurality of the second image acquisition devices;

[0032] The target panoramic image is determined according to the stitching seam position, the first environment image and the second environment image.

[0033] Optionally, determining the target panoramic image according to the stitching seam position, the first environment image, and the second environment image includes:

[0034] splicing the plurality of first environment images according to the splicing seam positions to obtain a candidate panoramic image;

[0035] The plurality of second environment images are fused with the candidate panoramic image according to the stitching seam positions to obtain the target panoramic image.

[0036] According to a second aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect of the embodiment of the present disclosure are implemented.

[0037] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0038] a memory having a computer program stored thereon;

[0039] A processor is used to execute the computer program in the memory to implement the steps of the method described in the first aspect of the embodiment of the present disclosure.

[0040] According to a fourth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect of the embodiment of the present disclosure.

[0041] According to a fifth aspect of an embodiment of the present disclosure, a vehicle is provided, wherein the vehicle is used to execute the steps of the method described in the first aspect of an embodiment of the present disclosure.

[0042] Through the above technical solution, the present disclosure first obtains the environmental information of the target area, and determines the distribution information of obstacles in the target area based on the environmental information. Among them, the target area is the detection area corresponding to at least one of the four corners of the vehicle body, and the distribution information represents the number of obstacles and the position of each obstacle. Then, according to the number of obstacles and the position of each obstacle, the stitching seam position of the target area is determined, and the target panoramic image corresponding to the vehicle is determined according to the stitching seam position. The present disclosure dynamically determines the stitching seam position according to the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. For the situation where there are multiple obstacles in the stitching area, the stitching seam position can also avoid obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam, and the panoramic image obtained is more accurate.

[0043] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide further understanding of the present disclosure and constitute a part of the specification. They are used to explain the present disclosure together with the following detailed description, but do not constitute a limitation of the present disclosure.

[0045] Figure 1 It is a schematic diagram of the layout of the cameras on the side of the vehicle body.

[0046] Figure 2 The figure is a flowchart of a method for determining an image according to an exemplary embodiment.

[0047] Figure 3 is based on Figure 2 A schematic diagram of ultrasonic ranging is shown in an embodiment.

[0048] Figure 4 The figure is a flowchart of another method for determining an image according to an exemplary embodiment.

[0049] Figure 5 is based on Figure 4 A schematic diagram of a detection area of ​​an ultrasonic radar shown in an embodiment.

[0050] Figure 6 The figure is a flowchart of another method for generating an image according to an exemplary embodiment.

[0051] Figure 7 The present invention is a flowchart of generating a panoramic image of a vehicle according to an exemplary embodiment.

[0052] Figure 8 The figure is a block diagram of a device for determining an image according to an exemplary embodiment.

[0053] Fig. 9 It is a block diagram of another apparatus for determining an image according to an exemplary embodiment.

[0054] Fig.10 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0055] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0056] Before introducing an image determination method, storage medium, electronic device, program product, and vehicle shown in an embodiment of the present disclosure, an application scenario of the embodiment of the present disclosure is first introduced.

[0057] like Figure 1 As shown, wide-angle cameras are arranged at the front, rear, left and right sides of the vehicle body. For example, the front camera is arranged at A near the front bumper, the rear camera is arranged at C near the rear bumper, and the left and right cameras are arranged at D and B on the sides of the vehicle body respectively. After the four wide-angle cameras capture images, they are spliced ​​into a panoramic surround image, in which the four corners of the top view are usually the splicing areas (Zone 1, Zone 2, Zone 3, Zone 4). When adjacent wide-angle images are spliced ​​in the splicing area, a splicing seam will appear. In the related art, obstacles (such as vehicles, pedestrians, etc.) located at the splicing seam cannot be displayed in the spliced ​​panoramic image, resulting in low accuracy of the generated panoramic image.

[0058] The present invention dynamically determines the position of the stitching seam according to the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. When there are multiple obstacles in the stitching area, the position of the stitching seam can also be made to avoid the obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam and obtaining a target panoramic image with higher accuracy.

[0059] Figure 2 is a flowchart of a method for determining an image according to an exemplary embodiment. Figure 2 As shown, the method may include the following steps.

[0060] Step S101, obtaining environmental information of a target area, where the target area is a detection area corresponding to at least one of the four corners of a vehicle body.

[0061] For example, multiple detection devices may be respectively arranged at the four corners of the vehicle body, wherein the detection devices may include radar detection devices and / or image detection devices, the radar detection devices may be, for example, ultrasonic radars, laser radars, etc., the image detection devices may be, for example, cameras, and the cameras may be, for example, monocular cameras, binocular cameras, etc. Each of the four corners of the vehicle body corresponds to a detection area, and the detection areas of the multiple detection devices at the four corners of the vehicle body may completely cover the image stitching areas at the four corners of the vehicle body. The wide-angle images collected by the cameras on the side of the vehicle body are stitched in the image stitching area to obtain a panoramic image of the vehicle.

[0062] In some embodiments, the target area can be first determined from the four detection areas corresponding to the four corners of the vehicle body. In a possible implementation, the target area can be determined according to the driving state and gear information of the vehicle. For example, when the vehicle is in the D gear, the detection areas corresponding to the two corners of the front side of the vehicle body can be used as the target area. When the vehicle is in the driving state and in the R gear, the detection areas corresponding to the two corners of the rear side of the vehicle body can be used as the target area.

[0063] Step S102: determining the distribution information of obstacles in the target area according to the environmental information, where the distribution information represents the number of obstacles and the position of each obstacle.

[0064] For example, taking the radar detection device as an ultrasonic radar and the image detection device as a camera, when the ultrasonic radar and the cameras at the four corners of the vehicle detect obstacles in the stitching area, the camera can collect obstacle images and extract obstacle instances through an instance segmentation algorithm. The ultrasonic radar can determine the obstacle coordinates using the triangulation positioning method through the detection distance information of multiple ultrasonic radars corresponding to different sub-areas.

[0065] The main ultrasonic radar ranging methods include phase detection method, transit time detection method, sound wave amplitude detection method, etc. Among them, the transit time detection method starts timing t0 when the ultrasonic sensor transmits the ultrasonic wave. The ultrasonic wave is reflected after hitting an obstacle. The ultrasonic receiver stops timing t1 when it receives the reflected wave. The distance between the obstacle and the ultrasonic radar is determined by the flight time Δt=t1-t0 between the received wave and the transmitted wave, and the ultrasonic propagation speed v.

[0066] like Figure 3 As shown in (a), the related technology cannot accurately describe the location of obstacles using a single ultrasonic radar. When obstacles are detected at different positions at the same distance from the ultrasonic radar, only the obstacle detection distance can be obtained, but the specific position cannot be confirmed. In addition, the information collected by the front and left wide-angle cameras in the edge area of ​​the image is severely distorted, and obstacles cannot be accurately identified even after correction. Figure 3The coordinates of the obstacle are determined by the positioning method shown in (b) in the figure. The obstacle position is confirmed by triangulation based on the detection distance information of the two ultrasonic radars to the same obstacle. The pixel areas corresponding to different partitions of the splicing area in the image can be confirmed by calibration, and the transformation relationship between the ultrasonic radar and the camera can be confirmed. The coordinates of the obstacle can be converted from the camera coordinate system to the radar coordinate system, thereby associating the obstacle instance determined by the camera with the obstacle position determined by the ultrasonic radar.

[0067] Step S103, determining the position of the seam of the target area according to the number of obstacles and the position of each obstacle.

[0068] For example, if there are no obstacles in the target area, the preset position can be used as the joint seam position. The preset position can be the position of the central axis of the target area, or the position of the central axis at a specified angle on both sides, for example, 5 degrees to the left of the central axis or 10 degrees to the right of the central axis, which is not specifically limited in the present disclosure. If there are obstacles in the target area, the joint seam position of the target area can be further determined according to the number of obstacles and the position of each obstacle.

[0069] In some embodiments, the detection area corresponding to the four corners of the vehicle body can be pre-divided into multiple sub-areas, such as Figure 3 As shown, for example, Zone1 can be divided into three sub-areas, L1, L2, and L3. First, the target sub-area can be determined from multiple sub-areas based on the number of obstacles and the position of each obstacle. If there is an obstacle in the target area and the obstacle is in motion, then the sub-area that is opposite to the obstacle's motion direction and farthest from the obstacle can be used as the target sub-area. If there is an obstacle in the target area and the obstacle is stationary, then the sub-area farthest from the obstacle can be used as the target sub-area. If there are multiple obstacles, then the target obstacle closest to the vehicle can be determined, and the sub-area farthest from the target obstacle can be used as the target sub-area. If there are multiple obstacles, the sub-area with the smallest obstacle density can also be used as the target sub-area.

[0070] In other embodiments, a preset position in the target sub-region may be used as the seam position, or the seam position may be dynamically determined in the target sub-region according to the position of the obstacle, so that the seam position avoids the obstacle as much as possible.

[0071] Step S104, determining the target panoramic image corresponding to the vehicle according to the stitching seam position.

[0072] For example, wide-angle images captured by cameras on the side of the vehicle body can be stitched together according to the stitching seam position to obtain a panoramic image of the vehicle. Since the stitching seam position is dynamically determined based on the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle, even if there are multiple obstacles in the image stitching area, whether the obstacles are scattered or densely distributed, the stitching seam position can be ensured to avoid obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam and obtaining a more accurate target panoramic image.

[0073] In summary, the present disclosure first obtains environmental information of the target area, and determines the distribution information of obstacles in the target area based on the environmental information. The target area is the detection area corresponding to at least one of the four corners of the vehicle body, and the distribution information represents the number of obstacles and the position of each obstacle. Then, based on the number of obstacles and the position of each obstacle, the position of the stitching seam in the target area is determined, and the target panoramic image corresponding to the vehicle is determined based on the stitching seam position. The present disclosure dynamically determines the stitching seam position based on the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. For the situation where there are multiple obstacles in the stitching area, the stitching seam position can also avoid obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam, and the panoramic image obtained is more accurate.

[0074] Figure 4 is a flowchart of another method for determining an image according to an exemplary embodiment. Figure 4 As shown, the method also includes the following steps.

[0075] Step S105, obtaining the gear information of the vehicle.

[0076] Step S106: determining a target area from a plurality of designated areas according to the gear information.

[0077] Step S107: Using the preset position as the stitching seam position of other areas, where the other areas include areas other than the target area among the multiple designated areas.

[0078] For example, different vehicle gears may correspond to different target areas, and the target area that needs to dynamically adjust the seam may be screened from multiple designated areas according to the gear information of the vehicle, and the seam position of the target area may be determined according to the number of obstacles and the position of each obstacle. The preset position may be used as the seam position of other areas in the multiple designated areas except the target area. The designated area may include four detection areas corresponding to the four corners of the vehicle body.

[0079] When the vehicle is in motion and in D gear, the detection areas corresponding to the two corners of the front side of the vehicle body can be used as target areas, and the position of the splicing seam of the detection areas corresponding to the two corners of the rear side of the vehicle body can be used as a preset position. When the vehicle is in motion and in R gear, the detection areas corresponding to the two corners of the rear side of the vehicle body can be used as target areas, and the position of the splicing seam of the detection areas corresponding to the two corners of the front side of the vehicle body can be used as a preset position. When the vehicle is in P gear or N gear, the detection areas at the four corners of the vehicle body can be used as target areas. In this way, the target area where the splicing seam position needs to be dynamically adjusted is determined according to the gear information, and the preset position is used as the splicing seam position of other areas. There is no need to dynamically adjust the splicing seam position of each image splicing area, which reduces the amount of data processing.

[0080] In some embodiments, each target area includes a plurality of sub-areas, and each sub-area is covered by the detection areas of at least two radar detection devices.

[0081] In a possible implementation, ultrasonic radars can be arranged at the four corners of the vehicle body to detect obstacles and their density. For example, 16 ultrasonic radars are arranged at the four corners of the vehicle body, 4 ultrasonic radars are arranged near the arc line of each vehicle body corner, and ultrasonic radars a, b, c, and d are arranged at the left front corner respectively, and the detection range of the 4 ultrasonic radars can completely cover the left front image stitching area.

[0082] like Figure 5 As shown in the figure, ultrasonic radars a and d are located at the outermost side, and the detection boundary needs to exceed the boundary line of the stitching area; ultrasonic radars b and c are located at the inner side, where ultrasonic radar b is close to radar a and its detection direction is biased towards the front of the vehicle (the axis in the detection area is biased towards l1), and ultrasonic radar c is close to radar d and its detection direction is biased towards the left side of the vehicle (the axis in the detection area is biased towards l2). The detection overlap area of ​​the four ultrasonic radars at each corner of the vehicle needs to completely cover the wide-angle image stitching area, such as Figure 5 As shown in the figure, the image stitching area Zone 1 can be divided into three areas, L1, L2, and L3, according to the detection overlap of ultrasonic radars a, b, c, and d. Among them, area L1 is the detection overlap area of ​​ultrasonic radar a and radar c, area L2 is the detection overlap area of ​​ultrasonic radar b and radar d, and area L3 is the detection overlap area of ​​ultrasonic radar b and radar c.

[0083] In practical applications, the number of radars may not necessarily be four, but may be determined based on the actual detection range of the ultrasonic radar used, so that each sub-area of ​​the image stitching area can be detected by at least two ultrasonic radars at the same time.

[0084] In some embodiments, when the number of obstacles is zero, that is, there are no obstacles in the target area, the preset position can be used as the seam position.

[0085] In other embodiments, when the number of obstacles is greater than or equal to one, that is, there are obstacles in the target area, the target sub-area can be determined from multiple sub-areas according to the position of the obstacle, and the specified position in the target sub-area is used as the seam position.

[0086] For example, if there is an obstacle in the target area, the direction of movement of the obstacle can be determined according to the position of the obstacle, and the sub-area that is opposite to the direction of movement of the obstacle and farthest from the obstacle is used as the target sub-area. Figure 5 As shown, if the obstacle is located in area L3 and the moving direction is toward area L1, then area L2 can be used as the target sub-area. The specified position can be the central axis of the target sub-area, or a position where the central axis deviates to the left or right by a specified angle, which is not specifically limited in the present disclosure.

[0087] If the number of obstacles is greater than one, that is, there are multiple obstacles, then the distribution state of the obstacles can be determined according to the positions of the obstacles, and the target sub-area can be determined according to the distribution state and the positions of the obstacles. The distribution state indicates whether the obstacles are dispersedly distributed or densely distributed.

[0088] If the distribution state characterizes that the obstacles are dispersed, then the target obstacle closest to the vehicle can be determined, and the sub-area farthest from the target obstacle can be used as the target sub-area. Figure 5 As shown, if obstacles A, B, and C are distributed in areas L1, L2, and L3 respectively, where the distance between obstacle A and the vehicle is 1m, the distance between obstacle B and the vehicle is 0.5m, and the distance between obstacle C and the vehicle is 1.2m, then obstacle B can be taken as the target obstacle, and area L1 farthest from obstacle B can be taken as the target sub-area.

[0089] If the distribution state characterizes that the obstacles are densely distributed, then the target obstacle closest to the vehicle can be determined. If there are multiple target obstacles closest to the vehicle, and the multiple target obstacles are located in different sub-areas, then the sub-area with the least number of obstacles can be used as the target sub-area. Figure 5 As shown, if the obstacles are densely distributed, there are 6 obstacles in area L1, 4 obstacles in area L2, and 5 obstacles in area L3, and there are target obstacles closest to the vehicle in areas L1, L2, and L3, then area L2 with the least number of obstacles can be used as the target sub-area.

[0090] In this way, the position of the stitching seam is dynamically determined according to the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. Even if there are multiple obstacles in the image stitching area, no matter whether the obstacles are scattered or densely distributed, the position of the stitching seam can be ensured to avoid the obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam and obtaining a more accurate target panoramic image.

[0091] Figure 6 is a flowchart of another method for generating an image according to an exemplary embodiment. Figure 6 As shown, step S104 can be implemented by the following steps.

[0092] Step S1041, acquiring multiple first environment images through multiple first image acquisition devices.

[0093] Step S1042: Acquire multiple second environment images through multiple second image acquisition devices.

[0094] Step S1043, determining a target panoramic image according to the stitching seam position, the first environment image and the second environment image.

[0095] For example, at least one first image acquisition device for acquiring images of the surrounding environment of the vehicle may be respectively disposed on the four sides of the vehicle, and at least one second image acquisition device for acquiring images of the surrounding environment corresponding to the four corners of the vehicle may be respectively disposed on the four corners of the vehicle. Figure 1 , the environmental images corresponding to the four corners of the vehicle body can be the environmental images of the stitching zones Zone 1, Zone 2, Zone 3, and Zone 4, respectively. The first image acquisition device can acquire multiple first environmental images corresponding to the four sides of the vehicle, and the second image acquisition device can acquire multiple second environmental images corresponding to the four corners of the vehicle body. The multiple second environmental images can be spliced ​​together according to the position of the stitching seam and in combination with the multiple first environmental images to obtain a target panoramic image. Since the position of the stitching seam is dynamically adjusted according to the distribution information of the obstacles, it is possible to avoid the loss of objects at the stitching seam as much as possible, and the accuracy of the target panoramic image is higher.

[0096] In some embodiments, multiple second environment images may be first stitched together according to the stitching seam positions to obtain a candidate panoramic image, and then it may be determined whether there are obstacles at the four stitching seam positions according to the first environment image. If there is an obstacle at a certain stitching seam position, the candidate panoramic image and the first environment image corresponding to the stitching seam may be fused to obtain a more accurate target panoramic image. In this way, even if the obstacles are densely distributed and there are obstacles at the dynamically adjusted stitching seam positions, it is possible to avoid the loss of objects at the stitching seams, and obtain a more accurate target panoramic image.

[0097] A specific embodiment is provided below. Figure 7 ,The process of generating a vehicle panoramic image is as shown below.

[0098] Step S01: Power on the vehicle, start the four wide-angle cameras for front, rear, left and right views, collect the four-way views around the vehicle, and simultaneously start the four-corner ultrasonic radar and four-corner cameras to obtain the environmental information of the panoramic stitching area at the four corners of the vehicle body.

[0099] Step S02: Input the default values ​​of the initial four-corner joint seams, that is, the preset positions.

[0100] Step S03: Determine the gear information, and the splicing area in the non-corresponding gear direction adopts the default value input in step S02.

[0101] Step S04: Based on the gear information of step S03, for the splicing area on the same side of the vehicle's driving direction as the gear, identify whether there is an obstacle in the detection area.

[0102] Step S05: If there is no obstacle in the joint area identified in step S04, the default value of the joint seam input in step S02 is still used.

[0103] Step S06: Determine whether the obstacle in the splicing area identified in step S04 is unique.

[0104] Step S07: If the obstacle in the joint area identified in step S06 is unique, the joint seam is dynamically adjusted to a partition without obstacles, or a sub-area farthest from the obstacle.

[0105] Step S08: If the obstacle in the joint area identified in step S06 is not unique, determine the distribution status of the obstacle.

[0106] Step S09: If the obstacles identified in step S08 are dispersed, the target obstacle closest to the vehicle is determined in combination with the ultrasonic radar ranging information, and the stitching seam is dynamically adjusted to the sub-area farthest from the target obstacle.

[0107] Step S10: If the obstacles identified in step S08 are densely distributed, the four-corner camera identification information and the ultrasonic radar ranging information are combined to dynamically adjust the stitching seam to the sub-area with the smallest obstacle density.

[0108] Step S11: based on the joint seam positions obtained in steps S05, S07, S09 and S10, determine the joint seam positions at the four corners of the vehicle body;

[0109] Step S12: combining the first environment image acquired by the four wide-angle cameras and the second environment image acquired by the four-corner cameras to obtain a panoramic surround image, that is, a target panoramic image.

[0110] In summary, the present disclosure first obtains environmental information of the target area, and determines the distribution information of obstacles in the target area based on the environmental information. The target area is the detection area corresponding to at least one of the four corners of the vehicle body, and the distribution information represents the number of obstacles and the position of each obstacle. Then, based on the number of obstacles and the position of each obstacle, the position of the stitching seam in the target area is determined, and the target panoramic image corresponding to the vehicle is determined based on the stitching seam position. The present disclosure dynamically determines the stitching seam position based on the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. For the situation where there are multiple obstacles in the stitching area, the stitching seam position can also avoid obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam, and the panoramic image obtained is more accurate.

[0111] Figure 8 is a block diagram of an image determination device according to an exemplary embodiment. Figure 8 As shown, the device 200 includes the following modules.

[0112] The first acquisition module 201 is configured to acquire environmental information of a target area, where the target area is a detection area corresponding to at least one of the four corners of the vehicle body.

[0113] The first determination module 202 is configured to determine the distribution information of obstacles in the target area according to the environmental information, where the distribution information represents the number of obstacles and the position of each obstacle.

[0114] The second determining module 203 is configured to determine the joint seam position of the target area according to the number of obstacles and the position of each obstacle.

[0115] The third determination module 204 is configured to determine the target panoramic image corresponding to the vehicle according to the stitching seam position.

[0116] Fig. 9 is a block diagram of another image determination device according to an exemplary embodiment. Fig. 9 As shown, the device 200 also includes the following modules.

[0117] The second acquisition module 205 is configured to acquire the gear information of the vehicle.

[0118] The fourth determination module 206 is configured to determine a target area from a plurality of designated areas according to the gear information, where the designated area includes detection areas corresponding to the four corners of the vehicle body.

[0119] In some embodiments, the second determining module 203 is further configured to: use the preset position as the stitching seam position of other areas, where the other areas include areas other than the target area in the plurality of designated areas.

[0120] In other embodiments, each target area includes multiple sub-areas, each sub-area is covered by the detection areas of at least two radar detection devices, and the second determination module 203 is configured to: when the number of obstacles is zero, use the preset position as the stitching seam position. Alternatively, when the number of obstacles is greater than or equal to one, determine the target sub-area from the multiple sub-areas according to the position of the obstacle, and use the specified position in the target sub-area as the stitching seam position.

[0121] In some other embodiments, the second determination module 203 is configured to: determine the movement direction of the obstacle according to the position of the obstacle when the number of obstacles is one, and determine the target sub-region in the sub-region opposite to the movement direction and farthest from the obstacle, the target sub-region being the sub-region farthest from the obstacle.

[0122] In some other embodiments, the second determination module 203 is configured to: when the number of obstacles is greater than one, determine the distribution state of the obstacles according to the positions of the obstacles, the distribution state characterizing whether the obstacles are dispersedly distributed or densely distributed, and determine the target sub-area according to the distribution state and the positions of the obstacles.

[0123] In some other embodiments, the second determination module 203 is configured to: determine the target obstacle closest to the vehicle when the distribution state characterizes that the obstacles are dispersedly distributed, and take the sub-region farthest from the target obstacle as the target sub-region.

[0124] In some other embodiments, the second determination module 203 is configured to: determine the target obstacle closest to the vehicle when the distribution state characterizes that the obstacles are densely distributed. When there are multiple target obstacles closest to the vehicle, and the multiple target obstacles are located in different sub-areas, the sub-area with the least number of obstacles is used as the target sub-area.

[0125] In other embodiments, a plurality of first image acquisition devices for acquiring images of the surrounding environment of the vehicle are disposed on the side of the vehicle, and at least one second image acquisition device for acquiring images of the environment corresponding to the four corners of the vehicle body is disposed at each of the four corners of the vehicle body. The third determination module 204 is configured to: acquire a plurality of first environment images through a plurality of first image acquisition devices. Acquire a plurality of second environment images through a plurality of second image acquisition devices. Determine the target panoramic image according to the position of the stitching seam, the first environment image, and the second environment image.

[0126] In some other embodiments, the third determination module 204 is configured to: stitch multiple first environment images according to the stitching seam position to obtain a candidate panoramic image, and fuse multiple second environment images with the candidate panoramic image according to the stitching seam position to obtain a target panoramic image.

[0127] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0128] In summary, the present disclosure first obtains environmental information of the target area, and determines the distribution information of obstacles in the target area based on the environmental information. The target area is the detection area corresponding to at least one of the four corners of the vehicle body, and the distribution information represents the number of obstacles and the position of each obstacle. Then, based on the number of obstacles and the position of each obstacle, the position of the stitching seam in the target area is determined, and the target panoramic image corresponding to the vehicle is determined based on the stitching seam position. The present disclosure dynamically determines the stitching seam position based on the number of obstacles in the target area corresponding to the four corners of the vehicle body and the position of each obstacle. For the situation where there are multiple obstacles in the stitching area, the stitching seam position can also avoid obstacles as much as possible, thereby avoiding the loss of objects at the stitching seam, and the panoramic image obtained is more accurate.

[0129] Fig.10 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Fig.10 As shown, the electronic device 300 may include: a processor 301 , a memory 302 . The electronic device 300 may also include one or more of a multimedia component 303 , an input / output (I / O) interface 304 , and a communication component 305 .

[0130] The processor 301 is used to control the overall operation of the electronic device 300 to complete all or part of the steps in the above-mentioned method for determining the image. The memory 302 is used to store various types of data to support the operation of the electronic device 300. For example, these data may include instructions for any application or method used to operate on the electronic device 300, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 302 or sent through the communication component 305. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 304 provides an interface between the processor 301 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 305 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0131] In an exemplary embodiment, the electronic device 300 may be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned image determination method.

[0132] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for determining an image are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 302 including program instructions, and the above-mentioned program instructions may be executed by the processor 301 of the electronic device 300 to complete the above-mentioned method for determining an image.

[0133] In another exemplary embodiment, a computer program product is further provided, comprising a computer program, which implements the steps of the above-mentioned image determination method when executed by a processor.

[0134] In another exemplary embodiment, a vehicle is also provided, wherein the vehicle is used to execute the steps of the above-mentioned image determination method.

[0135] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0136] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0137] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for determining an image, characterized in that: The method comprises: Acquire environmental information of a target area, wherein the target area is a detection area corresponding to at least one of the four corners of the vehicle body; a plurality of detection devices are respectively arranged at the four corners of the vehicle body, wherein the detection devices include a radar detection device and an image detection device, and the target area includes a plurality of sub-areas, each of which is covered by the detection areas of at least two of the radar detection devices; Determine distribution information of obstacles in the target area according to the environmental information, wherein the distribution information represents the number of obstacles and the position of each obstacle; Determining the position of the seam of the target area according to the number of the obstacles and the position of each obstacle; Determining a target panoramic image corresponding to the vehicle according to the stitching seam position; Determining the position of the seam of the target area according to the number of obstacles and the position of each obstacle includes: In the case that the number of the obstacles is greater than or equal to one, determining a target sub-region from the multiple sub-regions according to the positions of the obstacles, and using a designated position in the target sub-region as the joint seam position; The determining a target sub-area from the multiple sub-areas according to the position of the obstacle comprises: When the number of the obstacles is greater than one, determining the distribution state of the obstacles according to the positions of the obstacles, wherein the distribution state indicates whether the obstacles are dispersedly distributed or densely distributed; The target sub-area is determined according to the distribution state and the position of the obstacle.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining gear information of the vehicle; The target area is determined from a plurality of designated areas according to the gear information, and the designated areas include detection areas corresponding to four corners of a body of the vehicle.

3. The method according to claim 2, characterized in that The method further comprises: The preset position is used as the stitching seam position of other areas, and the other areas include areas of the multiple designated areas except the target area.

4. The method according to claim 1, characterized in that Determining the position of the seam of the target area according to the number of obstacles and the position of each obstacle includes: When the number of obstacles is zero, the preset position is used as the joint seam position.

5. The method according to claim 1, characterized in that The determining a target sub-area from the multiple sub-areas according to the position of the obstacle comprises: When the number of the obstacle is one, determining the movement direction of the obstacle according to the position of the obstacle; The target sub-region is determined from the sub-regions located in the direction opposite to the moving direction, and the target sub-region is the sub-region farthest from the obstacle.

6. The method according to claim 1, characterized in that The determining the target sub-area according to the distribution state and the position of the obstacle comprises: When the distribution state indicates that the obstacles are dispersedly distributed, determining a target obstacle closest to the vehicle; The sub-region farthest from the target obstacle is used as the target sub-region.

7. The method according to claim 1, characterized in that The determining the target sub-area according to the distribution state and the position of the obstacle comprises: When the distribution state indicates that the obstacles are densely distributed, determining a target obstacle closest to the vehicle; When there are multiple target obstacles closest to the vehicle and the multiple target obstacles are located in different sub-areas, the sub-area with the least number of obstacles is used as the target sub-area.

8. The method according to any one of claims 1 to 7, characterized in that The side of the vehicle is provided with a plurality of first image acquisition devices for acquiring images of the surrounding environment of the vehicle, and the four corners of the vehicle body are respectively provided with at least one second image acquisition device for acquiring images of the environment corresponding to the four corners of the vehicle body; Determining the target panoramic image corresponding to the vehicle according to the stitching seam position includes: Acquire a plurality of first environment images by using a plurality of the first image acquisition devices; Acquire a plurality of second environment images by using a plurality of the second image acquisition devices; The target panoramic image is determined according to the stitching seam position, the first environment image and the second environment image.

9. The method according to claim 8, characterized in that The determining the target panoramic image according to the stitching seam position, the first environment image and the second environment image comprises: splicing the plurality of first environment images according to the splicing seam positions to obtain a candidate panoramic image; The plurality of second environment images are fused with the candidate panoramic image according to the stitching seam positions to obtain the target panoramic image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

11. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

13. A vehicle, characterized in that: The vehicle is used to perform the steps of the method according to any one of claims 1-9.

Images