Image processing method and device, electronic equipment and vehicle

Abstract

The application discloses an image processing method and device, electronic equipment and vehicle. The method comprises the following steps: obtaining a first field of view image in a driving process of a first vehicle; determining at least one target image area based on the first field of view image, the target image area corresponding to a field of view blind area of the first vehicle in the first field of view image; obtaining a target field of view image collected by at least one target acquisition device of a second vehicle according to the target image area; wherein the relative position between at least the target acquisition device and the first vehicle is related to the relative position of the target image area in the first field of view image; and processing the target image area at least according to the target field of view image, so that the driving process of the first vehicle can refer to the first field of view image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an image processing method and device, electronic equipment and vehicle. BACKGROUND

[0002] In various scenarios, there are cases where multiple vehicles drive in the same range. For example, on a multi-lane road, two vehicles drive to the sidewalk and wait for pedestrians to pass.

[0003] Therefore, there are cases where vehicles cause each other's visual blind spots. SUMMARY

[0004] Therefore, the present application provides an image processing method and device, electronic equipment and vehicle, as follows:

[0005] An image processing method comprises:

[0006] obtaining a first visual field image during driving of a first vehicle;

[0007] determining at least one target image area based on the first visual field image, the target image area corresponding to a visual blind spot of the first vehicle in the first visual field image;

[0008] obtaining a target visual field image collected by a target collection device of at least one second vehicle according to the target image area, wherein the relative position between the target collection device and the first vehicle is related to the relative position of the target image area in the first visual field image;

[0009] processing the target image area according to at least the target visual field image, so that the driving process of the first vehicle can refer to the first visual field image.

[0010] The above method preferably comprises:

[0011] determining at least one target collection device from a plurality of image collection devices of the second vehicle according to the relative position of the target image area in the first visual field image, the image collection direction of the target collection device being related to the relative position of the target image area in the first visual field image;

[0012] obtaining an image collected by the target collection device to obtain a target visual field image.

[0013] The method, preferably, the plurality of image collection devices of the second vehicle comprises: an image collection device with a distance to the first vehicle less than or equal to a distance threshold; and the image collection device is a device configured with a super-bandwidth positioning structure.

[0014] The target image collection device is determined by the following method:

[0015] The position information of the plurality of image collection devices of the second vehicle is obtained by using a super-bandwidth positioning method.

[0016] At least one target image collection device is determined from the plurality of image collection devices according to the position information, and the position information of the target image collection device is related to the relative position of the target image collection device and the target image region in the first field-of-view image.

[0017] The method, preferably, the determination of at least one target image collection device from the plurality of image collection devices according to the position information comprises:

[0018] The attitude information of at least one initial image collection device from the plurality of image collection devices is obtained according to the position information; the attitude information at least represents the image collection direction of the initial image collection device; and the relative position between the initial image collection device and the first vehicle is related to the relative position of the target image region in the first field-of-view image.

[0019] At least one target image collection device is determined from the at least one initial image collection device according to the attitude information, and the attitude information of the target image collection device is related to the relative position of the target image collection device and the target image region in the first field-of-view image.

[0020] The method, preferably, the target field-of-view image collected by at least one target image collection device of the second vehicle is obtained according to the target image region, and the method comprises:

[0021] The second field-of-view image of at least one second vehicle is obtained; and the relative position between the second vehicle and the first vehicle is related to the relative position of the target image region in the first field-of-view image.

[0022] The target field-of-view image corresponding to at least one target image collection device is obtained in the second field-of-view image; the target image collection device is an image collection device arranged on the second vehicle, and the image collection direction of the target image collection device is related to the relative position of the target image region in the first field-of-view image.

[0023] The method, preferably, obtains a second view image of at least one second vehicle, comprising:

[0024] The position information of at least one neighboring vehicle of the first vehicle is obtained by using an ultra-wideband-based positioning method.

[0025] According to the position information of the neighboring vehicle, at least one second vehicle is determined in the neighboring vehicle.

[0026] The second view image transmitted by the second vehicle is obtained.

[0027] The method, preferably, processes the target image region according to at least the target view image, comprising:

[0028] The target view image and the first view image are fused so that the target image region is filled with at least part of the image region of the target view image.

[0029] An image processing apparatus, comprising:

[0030] A first obtaining unit is configured to obtain a first view image during driving of a first vehicle.

[0031] A region determining unit is configured to determine at least one target image region based on the first view image, the target image region corresponding to a view blind area of the first vehicle in the first view image.

[0032] A target obtaining unit is configured to obtain a target view image of at least one second vehicle collected by a target collection device according to the target image region, wherein a relative position between at least the target collection device and the first vehicle is related to a relative position of the target image region in the first view image.

[0033] A region processing unit is configured to process the target image region according to at least the target view image, so that the driving process of the first vehicle can refer to the first view image.

[0034] An electronic device, comprising:

[0035] A memory is configured to store an application program and data generated by running of the application program.

[0036] The processor is configured to execute the application program to: obtain a first field of view image in a driving process of a first vehicle; determine at least one target image region based on the first field of view image, the target image region corresponding to a blind area of the first vehicle in the first field of view image; obtain a target field of view image collected by a target acquisition device of at least one second vehicle according to the target image region, wherein a relative position between at least the target acquisition device and the first vehicle is related to a relative position of the target image region in the first field of view image; and process the target image region according to at least the target field of view image, so that the driving process of the first vehicle can refer to the first field of view image.

[0037] A vehicle as a first vehicle, comprising at least:

[0038] An image acquisition device configured to acquire an image;

[0039] An image display device;

[0040] A processor configured to obtain a first field of view image in a driving process of the first vehicle; determine at least one target image region based on the first field of view image, the target image region corresponding to a blind area of the first vehicle in the first field of view image; obtain a target field of view image collected by a target acquisition device of at least one second vehicle according to the target image region, wherein a relative position between at least the target acquisition device and the first vehicle is related to a relative position of the target image region in the first field of view image; process the target image region according to at least the target field of view image, and output the obtained first field of view image through the image display device, so that the driving process of the first vehicle can refer to the first field of view image.

[0041] It can be seen from the technical solution that, in the image processing method, device and electronic equipment disclosed in the application, after the first field of view image in the driving process of the first vehicle is obtained, the target image region corresponding to the blind area of the field of view of the first vehicle in the first field of view image is determined based on the first field of view image, and then the target field of view image collected by the target collection device of the second vehicle is obtained according to the target image region. The relative position between the target collection device and the first vehicle is related to the relative position of the target image region in the first field of view image, so that after the target image region is processed according to the target field of view image, the obtained first field of view image can be provided as a driving reference for the first vehicle in the driving process. It can be seen that, in the application, the field of view image of the first vehicle is processed by using the image corresponding to the blind area of the field of view of the first vehicle on the second vehicle, so that the blind area of the field of view image of the first vehicle is improved, thereby providing a reference for the first vehicle, thereby improving the driving experience of the vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0043] Figure 1 A flowchart of an image processing method provided by the first embodiment of the application;

[0044] Figure 2 An example diagram of a car collecting a field of view image by a camera;

[0045] Figure 3 An example diagram of a car whose camera is blocked to generate a blind area;

[0046] Figure 4 Another example diagram of a car collecting a field of view image by a camera of another vehicle;

[0047] Figure 5 A partial flowchart of an image processing method provided by the first embodiment of the application;

[0048] Figure 6 Another example diagram of a car collecting a field of view image by a camera of another vehicle;

[0049] Figure 7 Another partial flowchart of an image processing method provided by the first embodiment of the application;

[0050] Figure 8A structural schematic diagram of an image processing device provided for Embodiment Two of the present application;

[0051] Figure 9 A structural schematic diagram of an electronic device provided for Embodiment Three of the present application;

[0052] Figure 10 A structural schematic diagram of a vehicle provided for Embodiment Four of the present application;

[0053] Figure 11 A flowchart applicable to processing visual field blind area in the process of driving a vehicle according to the present application;

[0054] Figure 12 An example diagram applicable to processing visual field blind area in the process of driving a vehicle according to the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0056] Reference Figure 1 As shown in the figure, an implementation flowchart of an image processing method provided for Embodiment One of the present application, which can be applied to an electronic device capable of image processing, and the electronic device can be configured in a vehicle, such as a balance car, a car, or a ship, etc. that can be automatically driven or driven by a user. The technical solution in the present embodiment is mainly used to obtain a visual field image with improved visual field blind area to provide a reference for the vehicle, and to improve the driving experience of the vehicle.

[0057] Specifically, the method in the present embodiment can include the following steps:

[0058] Step 101: obtaining a first visual field image in the process of driving a first vehicle.

[0059] The first vehicle can be provided with a plurality of image acquisition devices. The image acquisition devices can be RGB cameras for acquiring RGB images, infrared cameras for acquiring infrared images, or lidar sensors for acquiring radar images, so as to acquire the surrounding view images of the first vehicle. Each image acquisition device has a respective image acquisition direction, and the image acquisition directions of each image acquisition device are different. The image acquisition devices on the first vehicle are arranged around the body of the first vehicle. Thus, the image acquisition devices can acquire view images at any visual angle relative to the first vehicle. Since the arrangement positions and postures of each image acquisition device on the first vehicle are different, the view angles of the view images acquired by each image acquisition device are different. Based on this, the view images acquired by each image acquisition device can be spliced and de-overlapped to obtain a panoramic view image of the first vehicle during driving, which is referred to as a first view image. The first view image can be an RGB view image, an infrared view image, a radar view image, or the like, based on the type of the image acquisition device.

[0060] For example, as shown in FIG. 1, four RGB cameras (or lidar sensors) are arranged at the front, rear, left side and right side of the vehicle a body, respectively, to acquire view images at various angles around the vehicle a, and then the view images at these angles are spliced and de-overlapped to obtain a panoramic view image of the vehicle a. Figure 2

[0061] It should be noted that the driving process of the first vehicle can include the process of driving the first vehicle onto the road and traveling, and the process of driving the first vehicle onto the road and pausing on the road. For example, the vehicle a is driving on the road, or the vehicle a is driving to the intersection and pausing to wait for the traffic light.

[0062] Step 102: determining at least one target image region based on the first view image.

[0063] The target image region corresponds to a visual blind area of the first vehicle in the first view image. During the driving process of the first vehicle, there can be a situation of being blocked by a second vehicle. Based on this, there is a visual blind area in the first view image of the first vehicle that is blocked by the second vehicle and cannot obtain a view image.

[0064] Specifically, in this embodiment, the visual blind area in the first view image can be identified by image recognition and other processing of the first view image, and the target image region corresponding to the visual blind area can be determined. At this time, the target image region is actually the image region of the second vehicle blocking the first vehicle in the first view image.​

[0065] For example, such as Figure 3 As shown, car b is to the left of car a, and car c is to the right of car a. Due to the obstruction of car b and car c, there are multiple blind spots in the panoramic view image of car a, which correspond to image area x1 of car b and image area x2 of car c, respectively. Based on this, in this embodiment, the panoramic view image of car a is identified to determine the blind spot image areas x1 and x2 corresponding to these two blind spots.

[0066] It should be noted that the image regions contained in the first field-of-view image correspond to the relative positions of their respective image acquisition devices on the first vehicle. For example, if the left side of car a is completely obscured by car b, the front left and rear left of car a are partially obscured by car b, the right side of car a is completely obscured by car c, and the front right and rear right of car a are partially obscured by car c, then the panoramic field-of-view image of car a contains two blind spot image regions x1 and x2. Blind spot image region x1 is captured by the camera on the left side of car a, corresponding to the left, front left, and rear left of car a. Blind spot image region x2 is captured by the camera on the right side of car a, corresponding to the right, front right, and rear right of car a.

[0067] Step 103: Based on the target image region, obtain the target field-of-view image acquired by the target acquisition device of at least one second vehicle.

[0068] Among them, at least the relative position between the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field of view image.

[0069] It should be noted that the target acquisition device is an image acquisition device installed on the second vehicle that obstructs the first vehicle. The relative position between the target acquisition device and the first vehicle can be understood as the relative position of the target acquisition device on the second vehicle relative to the first vehicle. The relative position of the target image region in the first field of view image can be understood as the relative position of the blind spot corresponding to the target image region relative to the first vehicle. In other words, in this embodiment, based on the relative position of the target image region in the first vehicle, the target field of view image acquired by the target acquisition device on the second vehicle, which is at the same relative position to the first vehicle, is obtained. This ensures that the image content of the acquired target field of view image is correlated with the image content of the target image region.

[0070] For example, the left side of the car a is blocked by car b, the right side of the car a is blocked by car c, and correspondingly, the blind area image region x1 contained in the panoramic view image of car a corresponds to the left side, the left front and the left rear of car a, and the blind area image region x2 corresponds to the right side, the right front and the right rear of car a. Based on this, the target view image is collected by using the camera on car b at the left side of car a, and the target view image is collected by using the camera on car c at the right side of car a, as shown in FIG. 6. Figure 4

[0071] Step 104: processing the target image region according to at least the target view image, so that the driving process of the first vehicle can refer to the first view image.

[0072] In the embodiment, the target image region can be processed according to the image content in the target view image, so that the image region corresponding to the view blind area in the obtained first view image can be improved.

[0073] Specifically, in the embodiment, the target view image and the first view image can be fused to fill the target image region with at least part of the image region of the target view image, so that the image region corresponding to the view blind area of the first vehicle in the obtained first view image is completed, and there is no longer an image region blocked by the second vehicle.

[0074] For example, the left side of the car a is blocked by car b, the right side of the car a is blocked by car c, and correspondingly, the blind area image region x1 contained in the panoramic view image of car a corresponds to the left side, the left front and the left rear of car a, and the blind area image region x2 corresponds to the right side, the right front and the right rear of car a. Based on this, the target view image is collected by using the camera on car b at the left side of car a, and the target view image is collected by using the camera on car c at the right side of car a, as shown in FIG. 6.

[0075] ​It can be seen from the technical solution that in the image processing method provided by the embodiment one of the present application, after the first field of view image in the driving process of the first vehicle is obtained, the target image region corresponding to the blind area of the field of view of the first vehicle in the first field of view image can be determined based on the first field of view image, and then the target field of view image collected by the target collection device of the second vehicle is obtained according to the target image region. The relative position between the target collection device and the first vehicle is related to the relative position of the target image region in the first field of view image, so that after the target image region is processed according to the target field of view image, the obtained first field of view image can be provided as a driving reference for the first vehicle in the driving process. It can be seen that in the embodiment of the present application, the field of view image of the first vehicle is processed by the image corresponding to the blind area of the field of view of the first vehicle on the second vehicle, so that the field of view image with the improved blind area of the field of view is obtained, which is provided as a reference for the first vehicle, thereby improving the driving experience of the vehicle.

[0076] In an implementation manner, when the target field of view image collected by the target collection device of the at least one second vehicle is obtained according to the target image region in step 103, the following manner can be used, as shown in step 104: Figure 5

[0077] Step 501: According to the relative position of the target image region in the first field of view image, at least one target collection device is determined from the plurality of image collection devices of the second vehicle.

[0078] In addition to the relative position between the target collection device and the first vehicle being related to the relative position of the target image region in the first field of view image, the image collection direction of the target collection device is also related to the relative position of the target image region in the first field of view image.

[0079] Based on this, in the embodiment, the target collection device with the image collection direction related to the relative position of the target image region in the first field of view image is determined from the plurality of image collection devices of the second vehicle.

[0080] Step 502: Obtain the image collected by the target collection device to obtain the target field of view image.

[0081] In addition to the relative position between the target collection device and the first vehicle being related to the relative position of the target image region in the first field of view image, the image collection direction of the target collection device is also related to the relative position of the target image region in the first field of view image.

[0082] ​For example, the left side of the vehicle a is blocked by the vehicle b, the right side of the vehicle a is blocked by the vehicle c, the blind area image region x1 contained in the panoramic view image of the vehicle a corresponds to the left side, the left front and the left rear of the vehicle a, the blind area image region x2 corresponds to the right side, the right front and the right rear of the vehicle a, the target view image is collected by using the camera on the vehicle b at the left side of the vehicle a and the collection direction is related to the left side, the left front and the left rear, the target view image is collected by using the camera on the vehicle c at the right side of the vehicle a and the collection direction is related to the right side, the right front and the right rear, that is, the target view image is collected by using the front, rear and left side cameras on the vehicle b and the target view image is collected by using the front, rear and right side cameras on the vehicle c, as shown in FIG. 8A, and then the panoramic view image of the vehicle a is image fused by using the target view images, so as to complete the blind area image region in the panoramic view image of the vehicle a blocked by the vehicle b and the vehicle c, so that the panoramic view image of the vehicle a can be provided to the driving process of the vehicle a, such as obstacle monitoring in the automatic driving process or human obstacle judgment in the user driving process. Figure 6

[0083] In a specific implementation, the multiple image collection devices of the second vehicle include: an image collection device on the second vehicle and the distance between the image collection device and the first vehicle is less than or equal to a distance threshold, and the image collection device is a device configured with an ultra wide band (UWB) positioning structure. That is, the image collection device on the second vehicle is a device supporting UWB function, so that the image collection device on the second vehicle can be communicated with and the image collected by the image collection device can be obtained based on the UWB positioning mode on the first vehicle.

[0084] Based on this, the target collection device on the second vehicle in the embodiment can be determined by the following method:

[0085] ​First, the position information of the image collection devices on the second vehicle is obtained by using the UWB-based positioning method. Specifically, the UWB positioning structure on the first vehicle sends a sinusoidal carrier to scan the image collection devices within a distance threshold from the first vehicle. Among these image collection devices, the image collection devices on the second vehicle are determined, and the UWB devices on other vehicles or objects are removed. Then, the UWB positioning structure on the first vehicle is used to position the scanned image collection devices on the second vehicle to obtain the position information of the image collection devices on the second vehicle. For example, the UWB positioning structure on the car a is used to position the UWB-enabled devices in the surrounding area, thereby obtaining the position information of the UWB-enabled cameras on the car b and the car c. The position information can be represented by latitude and longitude coordinates or coordinates with the first vehicle as the coordinate origin.

[0086] Then, at least one target collection device is determined from the image collection devices on the second vehicle based on the position information of the image collection devices. The position information of the target collection devices is related to the relative position of the image collection direction of the target collection devices and the target image region in the first field-of-view image.

[0087] In some embodiments, the image collection direction of the image collection devices on the second vehicle is preset and fixed based on their respective position information. Based on this, the position information of each image collection device in this embodiment can be used to determine the image collection direction of each image collection device, thereby determining the target collection devices as the image collection devices whose image collection direction is related to the relative position of the target image region in the first field-of-view image.

[0088] For example, the left side of the car a is blocked by car b, the right side of the car a is blocked by car c, the blind area image region x1 contained in the panoramic view image of car a corresponds to the left side, the left front and the left rear of car a, the blind area image region x2 corresponds to the right side, the right front and the right rear of car a, then the positions of each camera on car b and car c are located on car a by the positioning method based on UWB, and then the target cameras on car b corresponding to the left side, the left front and the left rear of car a are determined based on the fixed collection directions of these cameras, such as the cameras on car b located at the left side, the front and the rear, and the target cameras on car c corresponding to the right side, the right front and the right rear of car a are determined, such as the cameras on car c located at the right side, the right front and the right rear, that is, the target cameras are determined, and then the target view images are collected by using these cameras, and then the panoramic view image of car a is image fused by using these target view images, so that the blind area image region in the panoramic view image of car a caused by being blocked by car b and car c is completed, so that the panoramic view image of car a can be provided to the driving process of car a, such as the obstacle monitoring of pedestrians in the automatic driving process or the artificial obstacle judgment in the user driving process.

[0089] In some embodiments, the image collection directions of the image collection devices on the second vehicle are not fixed, but can be adjusted. Based on this, in the present embodiment, the position information of each image collection device can be obtained, and the posture information transmitted by at least one initial collection device in the plurality of image collection devices is obtained, and the posture information at least represents the image collection direction of the initial collection device, so that in the present embodiment, the initial collection device corresponding to the relative position of the first vehicle and the relative position of the target image region in the first view image is first determined on the second vehicle, and then the respective posture information transmitted by the initial collection device is received by the communication method based on UWB; then, at least one target collection device can be determined from the obtained initial collection device according to the posture information, and the posture information of the target collection device makes the image collection direction of the target collection device correspond to the relative position of the target image region in the first view image.

[0090] For example, the left side of the vehicle a is blocked by the vehicle b, the right side of the vehicle a is blocked by the vehicle c, the blind area image region x1 contained in the panoramic view image of the vehicle a corresponds to the left side, the left front and the left rear of the vehicle a, the blind area image region x2 corresponds to the right side, the right front and the right rear of the vehicle a, then the positions of each camera on the vehicle b and the vehicle c are located on the vehicle a by the positioning method based on UWB, the initial camera on the vehicle b which is located at the left side, the left front and the left rear of the vehicle a and the initial camera on the vehicle c which is located at the right side, the right front and the right rear of the vehicle a are found first among these cameras, the posture information transmitted by these initial cameras through UWB is received, the target cameras on the vehicle b which are directed to the left side, the left front and the left rear of the vehicle a and the target cameras on the vehicle c which are directed to the right side, the right front and the right rear of the vehicle a are found according to the directions of the cameras represented by the posture information, i.e. the image collection directions, then the target view images are collected by using these cameras, and finally the panoramic view image of the vehicle a is fused by using these target view images, so that the blind area image regions in the panoramic view image of the vehicle a caused by the blocking of the vehicle b and the vehicle c are completed, and the panoramic view image of the vehicle a can be provided for the driving process of the vehicle a, such as obstacle monitoring in the automatic driving process or human obstacle judgment in the user driving process.

[0091] In another implementation manner, the target view image collected by the target collection device of the at least one second vehicle according to the target image region in step 103 can be obtained by the following manner, as shown in step 103 in the following: Figure 7

[0092] Step 701: obtaining the second view image of the at least one second vehicle.

[0093] Wherein, the relative position between the second vehicle and the first vehicle is related to the relative position of the target image region in the first view image.

[0094] That is, the second vehicles whose relative positions with the first vehicle are related to the relative position of the target image region in the first view image are located in this embodiment, so as to obtain the second view images of these second vehicles.

[0095] ​Specifically, in the embodiment, the position information of at least one neighboring vehicle of the first vehicle can be obtained by using the UWB-based positioning manner. For example, the UWB positioning structure on the first vehicle sends a sinusoidal carrier to scan the neighboring vehicles within a distance less than or equal to a distance threshold from the first vehicle, and then obtains the position information of the neighboring vehicles transmitted by the neighboring vehicles through the UWB-based communication manner to realize the positioning of the neighboring vehicles. Then, according to the position information of the neighboring vehicles, the second vehicles are determined from the neighboring vehicles, which have a relative position with the first vehicle related to the relative position of the target image region in the first field-of-view image, and then the second field-of-view images transmitted by the second vehicles through the UWB-based communication manner are obtained. The second field-of-view image is a panoramic field-of-view image on the second vehicle, and the specific obtaining manner can refer to the obtaining manner of the first field-of-view image of the first vehicle.

[0096] For example, the left side of the vehicle a is blocked by the vehicle b, the right side of the vehicle a is blocked by the vehicle c, the blind area image region x1 in the panoramic field-of-view image of the vehicle a corresponds to the left side, the left front and the left rear of the vehicle a, and the blind area image region x2 corresponds to the right side, the right front and the right rear of the vehicle a. Then, the vehicles b, c and d around the vehicle a are positioned on the vehicle a through the UWB-based positioning manner, the position information transmitted by the vehicles b, c and d is received, and then the vehicles b and c on the left side of the vehicle a and the vehicles c and d on the right side of the vehicle a are determined as the second vehicles, and then the panoramic field-of-view images transmitted by the vehicles b and c are received.

[0097] Step 802: In the second field-of-view image, a target field-of-view image corresponding to at least one target acquisition device is obtained.

[0098] The target acquisition device is an image acquisition device arranged on the second vehicle, and the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field-of-view image. Specifically, in the embodiment, the target acquisition device whose image acquisition direction is related to the relative position of the target image region in the first field-of-view image can be determined according to the position information (and the attitude information) of each image acquisition device on the second vehicle, and the specific implementation manner can refer to the corresponding content in the foregoing. Then, the second field-of-view image is segmented according to the target acquisition devices, and the segmented image is the image region acquired by the target acquisition devices, so that the target field-of-view image corresponding to the target acquisition device in the second field-of-view image is obtained.

[0099] For example, the positions of each camera on the car b and the car c are located by the UWB-based positioning method on the car a, and the poses sent by each camera are also received, and the target cameras on the car b are found to be directed to the left side, the left front and the left back of the car a, and the target cameras on the car c are found to be directed to the right side, the right front and the right back of the car a, so that the field of view images corresponding to the target cameras directed to the left side, the left front and the left back of the car a are cut out from the panoramic field of view image of the car b, and the field of view images corresponding to the target cameras directed to the right side, the right front and the right back of the car a are cut out from the panoramic field of view image of the car c. Then, the panoramic field of view image of the car a is image fused using the target field of view images, so that the blind area image region of the panoramic field of view image of the car a caused by the occlusion of the car b and the car c is completed, and the panoramic field of view image of the car a can be provided for the driving process of the car a, such as obstacle monitoring in the automatic driving process or artificial obstacle judgment in the user driving process.

[0100] Reference Figure 8 A structural schematic diagram of an image processing device provided in Embodiment Two of the present application is provided, which can be configured in an electronic device capable of image processing, and the electronic device can be configured in a vehicle, such as a balance car, a car or a ship, etc. which can be automatically driven or driven by a user. The technical solution in the present embodiment is mainly used to obtain a field of view image with a completed blind area to provide a reference for the vehicle, and improve the driving experience of the vehicle.

[0101] Specifically, the device in the present embodiment can include the following units:

[0102] The first obtaining unit 801 is configured to obtain a first field of view image in a first vehicle driving process;

[0103] The region determining unit 802 is configured to determine at least one target image region based on the first field of view image, and the target image region corresponds to a blind area of the first vehicle in the first field of view image;

[0104] The target obtaining unit 803 is configured to obtain a target field of view image collected by a target collection device of at least one second vehicle according to the target image region, and a relative position between at least the target collection device and the first vehicle is related to a relative position of the target image region in the first field of view image;

[0105] The region processing unit 804 is configured to process the target image region according to at least the target field of view image, so that the driving process of the first vehicle can refer to the first field of view image.

[0106] ​It can be seen from the technical solution that, in the image processing device provided by the second embodiment of the present application, after the first field-of-view image of the first vehicle during driving is obtained, the target image region corresponding to the blind area of the first vehicle in the first field-of-view image can be determined based on the first field-of-view image, and then the target field-of-view image collected by the target image acquisition device of the second vehicle is obtained according to the target image region. Here, the relative position between the target image acquisition device and the first vehicle is related to the relative position of the target image region in the first field-of-view image. Therefore, after the target image region is processed according to the target field-of-view image, the obtained first field-of-view image can be provided as a driving reference for the first vehicle during driving. It can be seen that, in the embodiment of the present application, the field-of-view image of the first vehicle is processed by using the image corresponding to the blind area of the first vehicle on the second vehicle, so that the blind area in the field-of-view image of the first vehicle is improved, thereby providing a reference for the first vehicle, and improving the driving experience of the vehicle.

[0107] In an implementation manner, the target obtaining unit 803 is specifically configured to: according to the relative position of the target image region in the first field-of-view image, determine at least one target image acquisition device from the multiple image acquisition devices of the second vehicle, the image acquisition direction of the target image acquisition device being related to the relative position of the target image region in the first field-of-view image; and obtain the image collected by the target image acquisition device to obtain a target field-of-view image.

[0108] Optionally, the multiple image acquisition devices of the second vehicle include: an image acquisition device with a distance from the first vehicle less than or equal to a distance threshold; and the image acquisition device is a device configured with an ultra-wideband positioning structure.

[0109] The target obtaining unit 803 determines the target collection device by the following manner when determining the target collection device: obtaining position information of a plurality of image collection devices of the second vehicle by using the super bandwidth-based positioning manner; and determining at least one target collection device from the plurality of image collection devices according to the position information, wherein the position information of the target collection device is related to the relative position of the image collection direction of the target collection device and the target image region in the first field of view image. For example, first, obtain attitude information sent by at least one initial collection device from the plurality of image collection devices according to the position information; the attitude information at least represents the image collection direction of the initial collection device; the relative position between the initial collection device and the first vehicle is related to the relative position of the target image region in the first field of view image; and then determine at least one target collection device from the at least one initial collection device according to the attitude information, wherein the attitude information of the target collection device is related to the relative position of the image collection direction of the target collection device and the target image region in the first field of view image.

[0110] In an implementation manner, the target obtaining unit 803 is specifically configured to: obtain a second field of view image of at least one second vehicle; the relative position between the second vehicle and the first vehicle is related to the relative position of the target image region in the first field of view image; and obtain a target field of view image corresponding to at least one target collection device in the second field of view image, wherein the target collection device is an image collection device arranged on the second vehicle, and the image collection direction of the target collection device is related to the relative position of the target image region in the first field of view image.

[0111] Optionally, when obtaining the second field of view image of at least one second vehicle, the target obtaining unit 803 is specifically configured to: obtain position information of at least one neighboring vehicle of the first vehicle by using the super bandwidth-based positioning manner; determine at least one second vehicle from the neighboring vehicles according to the position information of the neighboring vehicles; and obtain the second field of view image transmitted by the second vehicle.

[0112] In an implementation manner, the region processing unit 804 is specifically configured to: fuse the target field of view image and the first field of view image, so that the target image region is filled with at least part of the image region of the target field of view image.

[0113] It should be noted that the specific implementation of each unit in this embodiment can refer to the corresponding content in the foregoing, which will not be described in detail here.

[0114] Reference Figure 9A structural schematic diagram of an electronic device is provided in Embodiment Three of the present application. The electronic device can be an electronic device capable of image processing. The electronic device can be configured in a vehicle, such as a self-driving vehicle or a vehicle driven by a user, for example, a balance car, a car, or a ship. The technical solution in the present embodiment is mainly used to obtain a field of view image with improved blind area to provide a reference for the vehicle, thereby improving the driving experience of the vehicle.

[0115] Specifically, the electronic device in the present embodiment can include the following structure:

[0116] The memory 901 is configured to store application programs and data generated during running of the application programs.

[0117] The processor 902 is configured to execute the application programs to achieve the following: obtaining a first field of view image in a driving process of a first vehicle; determining at least one target image region based on the first field of view image, the target image region corresponding to a blind area of the first vehicle in the first field of view image; obtaining a target field of view image collected by a target collection device of at least one second vehicle according to the target image region; wherein a relative position between the target collection device and the first vehicle is related to a relative position of the target image region in the first field of view image; and processing the target image region at least according to the target field of view image, so that the driving process of the first vehicle can refer to the first field of view image.

[0118] As can be seen from the above technical solution, in the electronic device provided in Embodiment Three of the present application, after obtaining a first field of view image in a driving process of a first vehicle, a target image region corresponding to a blind area of the first vehicle in the first field of view image is determined based on the first field of view image. Then, a target field of view image collected by a target collection device of a second vehicle is obtained according to the target image region. Here, a relative position between the target collection device and the first vehicle is related to a relative position of the target image region in the first field of view image. Therefore, after processing the target image region according to the target field of view image, the obtained first field of view image can be provided to the first vehicle as a driving reference in the driving process. As can be seen, in the present embodiment, the field of view image of the first vehicle is processed by using the image corresponding to the blind area of the first vehicle on the second vehicle, so that the blind area in the field of view image of the first vehicle is improved, thereby providing a reference for the first vehicle, and improving the driving experience of the vehicle.

[0119] It should be noted that the specific implementation of the processor in the present embodiment can refer to the corresponding content in the foregoing description, which will not be described in detail here.

[0120] ReferenceFigure 10 A schematic diagram of a structure of a vehicle is provided in Embodiment Four of the present application. In addition to the engine and the moving device (wheels or propellers) and other structures, the vehicle as the first vehicle can also include the following structures:

[0121] A plurality of image acquisition devices 1001, such as cameras, are arranged on the periphery of the first vehicle to facilitate the acquisition of the field of view images at various angles of the first vehicle.

[0122] An image display device 1002, such as a display screen, is used to output images.

[0123] A processor 1003 is used to obtain a first field of view image of the first vehicle during driving according to the field of view images acquired by the image acquisition device 1001; determine at least one target image region based on the first field of view image, the target image region corresponding to a blind area of the first vehicle in the first field of view image; obtain a target field of view image acquired by a target acquisition device of at least one second vehicle according to the target image region; wherein the relative position between at least the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field of view image; process the target image region according to at least the target field of view image, and output the obtained first field of view image through the image display device 1002, so that the driving process of the first vehicle can refer to the first field of view image.

[0124] As can be seen from the above technical solution, in the vehicle provided in Embodiment Four of the present application, after obtaining the first field of view image of the first vehicle during driving, the target image region corresponding to the blind area of the first vehicle in the first field of view image can be determined based on the first field of view image, and then the target field of view image acquired by the target acquisition device of the second vehicle can be obtained according to the target image region. Here, the relative position between the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field of view image. Therefore, after processing the target image region according to the target field of view image, the obtained first field of view image can be provided to the first vehicle as a driving reference during driving. As can be seen, in the present application, the field of view image of the vehicle is processed by the image corresponding to the blind area of the vehicle on other vehicles, so that the blind area in the field of view image of the vehicle is improved, thereby providing a reference to the vehicle, thereby improving the driving experience of the vehicle.

[0125] Taking a vehicle driving on a road as an example, the technical solution of the present application is described in detail as follows:

[0126] First, the UWB positioning technology is used in the vehicle to realize accurate positioning between multiple cameras (or sensors such as laser radars), and the image fusion of the multiple cameras (or sensors such as laser radars) is realized by using the positioning information such as position or attitude, so as to solve the occlusion problem between vehicles.

[0127] Based on the above core technical content, combined with the flowchart shown in Figure 11 and the example diagram shown in Figure 12 , the following is the technical solution of the specific implementation:

[0128] 1. The camera on the vehicle and the occluded vehicle on the road supports UWB;

[0129] 2. By using the UWB positioning technology, the vehicle can realize the positioning of the adjacent vehicle-mounted camera;

[0130] 3. Obtain the image and attitude of the adjacent camera, fuse the adjacent camera photos into the field of view of the vehicle, and integrate the real-time panoramic road condition information of the vehicle, that is, the vehicle camera detects a vehicle in the left front field of view, then the camera of the vehicle in the left front field of view is called, and thus the image collected by the camera of the vehicle in the left front field of view is used to fill the occluded image in the left front field of view of the vehicle.

[0131] Based on this, in the technical solution of the present application, the occluded vehicle's field of view image is fused into the blind area of the vehicle by using the accurate camera positioning and attitude information, so as to detect whether there is a pedestrian in the field of view, so as to detect the early warning danger of the pedestrian crossing the road caused by the occlusion of the vehicle in the left front field of view. In summary, in the present application, the sensors (cameras, laser radars, etc.) of multiple vehicles in the adjacent area can be used to realize data sharing and synthesize the panoramic road condition of the adjacent area, so as to solve the problem of blind area detection.

[0132] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts are described in the method part.

[0133] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0134] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and

[0135] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, comprising: Obtain the first-view image during the driving process of the first vehicle; Based on the first field-of-view image, at least one target image region is determined, the target image region corresponding to the blind spot of the first vehicle in the first field-of-view image; Based on the target image region, a target field-of-view image acquired by at least one target acquisition device of a second vehicle is obtained; wherein, at least the relative position between the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field-of-view image; the target acquisition device is determined by: obtaining position information of multiple image acquisition devices of the second vehicle, and at least based on the position information, determining at least one target acquisition device among the multiple image acquisition devices, wherein the position information of the target acquisition device is such that the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field-of-view image; The target image region is processed based on at least the target field-of-view image so that the driving process of the first vehicle can refer to the first field-of-view image.

2. The method according to claim 1, wherein obtaining a target field-of-view image acquired by a target acquisition device of at least one second vehicle based on the target image region, comprises: Based on the relative position of the target image region in the first field of view image, at least one target acquisition device is determined among the multiple image acquisition devices of the second vehicle, wherein the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field of view image; The image acquired by the target acquisition device is obtained to obtain the target field-of-view image.

3. The method according to claim 2, wherein the plurality of image acquisition devices of the second vehicle comprises: An image acquisition device whose distance to the first vehicle is less than or equal to a distance threshold; The image acquisition device is a device equipped with an ultra-wideband positioning structure; The target acquisition device is determined in the following way: Using an ultra-wideband positioning method, the location information of multiple image acquisition devices of the second vehicle is obtained; Based at least the location information, at least one target acquisition device is determined among the plurality of image acquisition devices, wherein the location information of the target acquisition device is such that the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field of view image.

4. The method according to claim 3, wherein at least one target acquisition device is determined among the plurality of image acquisition devices based on the location information, comprising: Based on the location information, obtain the attitude information sent by at least one initial acquisition device among the plurality of image acquisition devices; The attitude information at least characterizes the image acquisition direction of the initial acquisition device; the relative position between the initial acquisition device and the first vehicle is related to the relative position of the target image region in the first field-of-view image; Based on the attitude information, at least one target acquisition device is determined among the at least one initial acquisition device, wherein the attitude information of the target acquisition device is such that the image acquisition direction of the target acquisition device corresponds to the relative position of the target image region in the first field of view image.

5. The method according to claim 1, wherein obtaining a target field-of-view image acquired by a target acquisition device of at least one second vehicle based on the target image region comprises: Obtain a second field-of-view image of at least one second vehicle; The relative position between the second vehicle and the first vehicle is related to the relative position of the target image region in the first field of view image; In the second field-of-view image, a target field-of-view image corresponding to at least one target acquisition device is obtained. The target acquisition device is an image acquisition device installed on the second vehicle, and the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field-of-view image.

6. The method of claim 5, wherein obtaining a second field-of-view image of at least one second vehicle comprises: Using an ultra-wideband positioning method, the location information of at least one neighboring vehicle of the first vehicle is obtained; Based on the location information of the nearby vehicles, at least one second vehicle is identified among the nearby vehicles; Obtain the second field-of-view image transmitted by the second vehicle.

7. The method according to claim 1, 2, or 5, wherein at least based on the target field-of-view image, the target image region is processed, including: The target field-of-view image and the first field-of-view image are fused together so that the target image region is filled with at least a portion of the image region of the target field-of-view image.

8. An image processing apparatus, comprising: The first acquisition unit is used to acquire a first field-of-view image during the driving process of the first vehicle; The region determination unit is configured to determine at least one target image region based on the first field-of-view image, the target image region corresponding to the blind spot of the first vehicle in the first field-of-view image; A target acquisition unit is configured to acquire a target field-of-view image captured by at least one target acquisition device of a second vehicle based on the target image region; wherein the relative position between at least the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field-of-view image; the target acquisition device is determined by: acquiring position information of multiple image acquisition devices of the second vehicle, and at least based on the position information, determining at least one target acquisition device among the multiple image acquisition devices, wherein the position information of the target acquisition device is such that the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field-of-view image; A region processing unit is configured to process the target image region based at least on the target field-of-view image, so that the driving process of the first vehicle can refer to the first field-of-view image.

9. An electronic device, comprising: Memory, used to store applications and the data generated by the running of the applications; A processor for executing the application to achieve: obtaining a first field-of-view image during the driving of the first vehicle; Based on the first field-of-view image, at least one target image region is determined, the target image region corresponding to the blind spot of the first vehicle in the first field-of-view image; Based on the target image region, a target field-of-view image acquired by at least one target acquisition device of a second vehicle is obtained; wherein, at least the relative position between the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field-of-view image; at least based on the target field-of-view image, the target image region is processed so that the driving process of the first vehicle can refer to the first field-of-view image; the target acquisition device is determined by: obtaining position information of multiple image acquisition devices of the second vehicle, and at least based on the position information, determining at least one target acquisition device among the multiple image acquisition devices, wherein the position information of the target acquisition device is such that the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field-of-view image.

10. A means of transport, wherein the means of transport, as a first means of transport, comprises at least: Image acquisition device, used to acquire images; Image display device; A processor is used to obtain a first field-of-view image during the driving process of the first vehicle; Based on the first field-of-view image, at least one target image region is determined, the target image region corresponding to the blind spot of the first vehicle in the first field-of-view image; Based on the target image region, a target field-of-view image acquired by a target acquisition device of at least one second vehicle is obtained; wherein, at least the relative position between the target acquisition device and the first vehicle is related to the relative position of the target image region in the first field-of-view image; at least based on the target field-of-view image, the target image region is processed, and the obtained first field-of-view image is output through the image display device, so that the driving process of the first vehicle can refer to the first field-of-view image. The target acquisition device is determined by: obtaining the location information of multiple image acquisition devices of the second vehicle; and, based on the location information, determining at least one target acquisition device among the multiple image acquisition devices, wherein the location information of the target acquisition device is such that the image acquisition direction of the target acquisition device is related to the relative position of the target image region in the first field of view image.

Images