Method and device for determining driving area information

Abstract

The embodiment of the invention provides a method and device for determining driving area information. The method comprises the following steps: acquiring first information; the first information comprises information of an initial driving area determined according to at least one image, and the at least one image comes from at least one camera module; acquiring second information; the second information comprises radar detection information; and determining first driving area information according to the first information and the second information. The first driving area information obtained through the method for determining the driving area information fuses the image information and the radar detection information, and the accuracy is higher. When one of the camera device and the radar device is inaccurate in detection, for example, the camera device is in a shadow or sheltered environment, or a laser radar is influenced by climate, a vehicle-mounted radar is influenced by noise signals and the like, the accuracy of the information of the first driving area can be improved through the detection information of the other one, and the safety of vehicle driving is ensured.

Description

[0001] This application is a divisional application. The application number of the original application is 201910525320.1, and the original application date is June 18, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of autonomous driving technology, and in particular to a method and device for determining driving area information. Background Art

[0003] The driving area of ​​a vehicle is the area in the driving environment in which the vehicle is located that can be driven by the vehicle. The driving area of ​​the vehicle must avoid other vehicles, pedestrians, non-vehicle driving areas on the road, and other obstacle areas in the driving environment in which the vehicle is located.

[0004] In the prior art, a camera device can be used to capture an image of the area to be detected in the driving environment where the vehicle is located, and then a semantic segmentation algorithm or an instance segmentation algorithm based on a convolutional neural network (CNN) is used to identify the image to determine the driving area in the area to be detected. However, since the camera device is greatly affected by the lighting conditions, it is easy to make a misjudgment in an environment with shadows or shielding, resulting in inaccurate driving area information. In addition, a laser radar can be used to emit a laser to the area to be detected or an on-board radar can be used to emit an electromagnetic wave to the area to be detected, and the obstacle target in the area to be detected can be determined by the echo, thereby determining the driving area in the area to be detected. However, since the laser radar is easily affected by the climate, the measurement accuracy of the laser radar will be reduced in extreme climates such as rain and snow, resulting in inaccurate driving area information; and the on-board radar is easily affected by the noise signal in the environment. When the amplitude of the noise signal exceeds the detection threshold, the on-board radar will misjudge the obstacle target and give a false alarm, or when there are multiple strong interference targets in the environment, the obstacle target will be missed, resulting in inaccurate driving area information. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining driving area information to solve the problem of inaccurate driving area information.

[0006] In a first aspect, an embodiment of the present application provides a method for determining driving area information, the method comprising: obtaining first information; the first information includes information of an initial driving area determined based on at least one image, and the at least one image comes from at least one camera module; obtaining second information; the second information includes radar detection information; determining the first driving area information based on the first information and the second information.

[0007] In this implementation, firstly, the first information is obtained, the first information includes the information of the initial driving area determined according to the image, then the second information including the radar detection information is obtained, and finally, the first driving area information is determined according to the first information and the second information. It can be seen that the first driving area information obtained by the method for determining the driving area information provided by this implementation integrates the image information and the radar detection information, and has higher accuracy. Secondly, when the camera device or the radar device has inaccurate detection, such as the camera device is in a shadow or blocked environment, or the laser radar is affected by the climate, and the vehicle-mounted radar is affected by the noise signal, etc., the accuracy of the first driving area information can be improved through the detection information of the other party to ensure the safety of vehicle driving.

[0008] In combination with the first aspect, in a first possible implementation manner of the first aspect, the radar detection information includes detection information of the initial driving area.

[0009] In this implementation, the detection information of the initial driving area included in the radar detection information is determined as the second information. The subsequent radar only needs to send the detection information of the initial driving area to the driving area information acquisition device. The data transmission volume is small, and when the first driving area information is subsequently determined based on the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0010] In combination with the first aspect, in a second possible implementation manner of the first aspect, the method further includes: determining whether a region of interest exists according to the first information and the second information.

[0011] In this implementation, by determining whether there is an area of ​​interest, the accuracy of the first driving area information can be further determined, thereby ensuring the accuracy of the determined first driving area information.

[0012] In combination with the first aspect, in a third possible implementation of the first aspect, the method also includes: determining the existence of an area of ​​interest; obtaining second driving area information and area information of the area of ​​interest; obtaining a detection result of the area of ​​interest based on the area information of the area of ​​interest; and determining the first driving area information based on the second driving area information and the detection result of the area of ​​interest.

[0013] In this implementation, when it is determined that there is an area of ​​interest, the information obtained after the fusion of the first information and the second information is determined as the second driving area information, and based on the area information of the area of ​​interest, the area of ​​interest is detected to obtain the detection result of the area of ​​interest, and finally, the first driving area information is determined based on the second driving area information and the detection result of the area of ​​interest, so that the determined first driving area information is more accurate, and subsequent vehicles are safer when driving according to the first driving area information.

[0014] In combination with the first aspect, in a fourth possible implementation of the first aspect, the information of the initial driving area includes information of at least one pixel point corresponding to the initial driving area; or, the information of the initial driving area is indicated by boundary information of the initial driving area.

[0015] In this implementation, the information of the initial driving area can be indicated by the boundary information of the initial driving area, so that when the information of the initial driving area is sent, the amount of data transmission can be reduced, thereby improving the efficiency of determining the driving area information.

[0016] In combination with the first aspect, in a fifth possible implementation manner of the first aspect, the radar detection information includes position coordinate information; or, the radar detection information includes position coordinate information and covariance information of the position coordinates.

[0017] In this implementation, the radar detection information may include position coordinate information and covariance information of the position coordinates, so that after the radar sends the radar detection information to the driving area information acquisition device, the driving area information acquisition device can more accurately determine the target point detected by the radar, thereby improving the accuracy of the first driving area information determined subsequently.

[0018] In second aspect, an embodiment of the present application provides a method for determining driving area information, the method comprising: receiving first information; the first information includes information of an initial driving area determined based on at least one image, and the at least one image comes from at least one camera module; generating second information; the second information includes radar detection information of the initial driving area generated based on the first information; and sending the second information.

[0019] In this implementation, firstly, the first information is received, then the second information including the radar detection information of the initial driving area is generated according to the first information, and finally the second information is sent to the driving area information acquisition device. It can be seen that, by adopting the method provided by this implementation, the radar only needs to send the radar detection information of the initial driving area to the driving area information acquisition device, and the data transmission volume is small. When the first driving area information is subsequently determined according to the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0020] In the third aspect, an embodiment of the present application provides a device for determining driving area information, the device comprising: an acquisition module for acquiring first information; the first information includes information of an initial driving area determined based on at least one image, and the at least one image comes from at least one camera module; the acquisition module is also used to acquire second information; the second information includes radar detection information; a determination module is used to determine the first driving area information based on the first information and the second information.

[0021] The device provided by this implementation method first obtains the first information, which includes the information of the initial driving area determined according to the image, and then obtains the second information including the radar detection information. Finally, the first driving area information is determined according to the first information and the second information. It can be seen that the first driving area information obtained by the device for determining the driving area information provided by this implementation method integrates the image information and the radar detection information, and has higher accuracy. Secondly, when the camera device or the radar device has inaccurate detection, such as the camera device is in a shadow or blocked environment, or the laser radar is affected by the climate, and the vehicle-mounted radar is affected by the noise signal, etc., the accuracy of the first driving area information can be improved through the detection information of the other party to ensure the safety of vehicle driving.

[0022] In combination with the third aspect, in a first possible implementation manner of the third aspect, the radar detection information includes detection information of the initial driving area.

[0023] The device provided by the present implementation method determines the detection information of the initial driving area included in the radar detection information as the second information. The subsequent radar only needs to send the detection information of the initial driving area to the driving area information acquisition device. The data transmission volume is small, and when the first driving area information is subsequently determined based on the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0024] In combination with the third aspect, in a second possible implementation manner of the third aspect, the determination module is further used to: determine whether there is a region of interest based on the first information and the second information.

[0025] The device provided in this implementation manner can further determine the accuracy of the first driving area information by determining whether there is an area of ​​interest, thereby ensuring the accuracy of the determined first driving area information.

[0026] In combination with the third aspect, in a third possible implementation method of the third aspect, the determination module is also used to determine the existence of an area of ​​interest; the acquisition module is also used to obtain second driving area information and area information of the area of ​​interest; the acquisition module is also used to obtain the detection result of the area of ​​interest based on the area information of the area of ​​interest; the determination module is also used to determine the first driving area information based on the second driving area information and the detection result of the area of ​​interest.

[0027] The device provided by the present implementation method, when determining that there is an area of ​​interest, determines the information obtained after fusing the first information and the second information as the second driving area information, and detects the area of ​​interest based on the area information of the area of ​​interest to obtain the detection result of the area of ​​interest, and finally determines the first driving area information based on the second driving area information and the detection result of the area of ​​interest, so that the accuracy of the determined first driving area information is higher, and subsequent vehicles are safer when driving according to the first driving area information.

[0028] In combination with the third aspect, in a fourth possible implementation of the third aspect, the information of the initial driving area includes information of at least one pixel point corresponding to the initial driving area; or, the information of the initial driving area is indicated by boundary information of the initial driving area.

[0029] In the device provided by the present implementation, the information of the initial driving area can be indicated by the boundary information of the initial driving area, so that when the information of the initial driving area is sent, the amount of data transmission can be reduced, thereby improving the efficiency of determining the driving area information.

[0030] In combination with the third aspect, in a fifth possible implementation manner of the third aspect, the radar detection information includes position coordinate information; or, the radar detection information includes position coordinate information and covariance information of the position coordinates.

[0031] In the device provided by the present implementation, the radar detection information may include position coordinate information and covariance information of the position coordinates, so that after the radar sends the radar detection information to the driving area information acquisition device, the driving area information acquisition device can more accurately determine the target point detected by the radar, thereby improving the accuracy of the first driving area information subsequently determined.

[0032] In a fourth aspect, an embodiment of the present application provides a device for determining driving area information, the device comprising: a receiving module for receiving first information; the first information includes information of an initial driving area determined based on at least one image, and the at least one image comes from at least one camera module; a generating module for generating second information; the second information includes radar detection information of the initial driving area generated based on the first information; and a sending module for sending the second information.

[0033] The device provided by this implementation first receives the first information, then generates the second information including the radar detection information of the initial driving area according to the first information, and finally sends the second information to the driving area information acquisition device. It can be seen that in the device provided by this implementation, the radar only needs to send the radar detection information of the initial driving area to the driving area information acquisition device, and the data transmission volume is small. When the first driving area information is subsequently determined according to the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0034] In a fifth aspect, an embodiment of the present application provides a device, comprising a processor, wherein the processor is used to couple with a memory, and the processor is used to read instructions in the memory so that the device executes the first aspect, various possible implementation methods of the first aspect, and the method in the second aspect according to the instructions.

[0035] In a sixth aspect, an embodiment of the present application provides a computer storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes part or all of the steps of the method in the first aspect, various possible implementations of the first aspect, and the second aspect.

[0036] In the seventh aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables the computer to execute part or all of the steps of the method in the first aspect, various possible implementations of the first aspect, and the second aspect.

[0037] In order to solve the problem of inaccurate driving area information obtained, an embodiment of the present application provides a method and device for determining driving area information. In the method, firstly, the first information is obtained, and the first information includes information of the initial driving area determined according to the image, and then the second information including radar detection information is obtained, and finally, the first driving area information is determined according to the first information and the second information. It can be seen that the first driving area information obtained by the method for determining the driving area information provided by the present application integrates the image information and the radar detection information, and has higher accuracy. Secondly, when the camera device or the radar device has inaccurate detection, for example, the camera device is in a shadowed or blocked environment, or the laser radar is affected by the climate, and the vehicle-mounted radar is affected by the noise signal, etc., the accuracy of the first driving area information can be improved through the detection information of the other party to ensure the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A structural block diagram of an implementation of a system for determining driving area information provided by the present application;

[0039] Figure 2 A structural block diagram of another embodiment of the system for determining driving area information provided by the present application;

[0040] Figure 3 A flowchart of an implementation method of the method for determining driving area information provided by the present application;

[0041] Figure 4 A flowchart of an implementation method of the method for obtaining first information provided in the present application;

[0042] Figure 5 A flowchart of another implementation of the method for obtaining first information provided in the present application;

[0043] Figure 6 A flowchart of another implementation of the method for obtaining first information provided in the present application;

[0044] Figure 7 A flowchart of an implementation method of the method for obtaining second information provided in the present application;

[0045] Figure 8 A flowchart of another implementation of the method for obtaining second information provided by the present application;

[0046] Fig. 9 A flowchart of another implementation of the method for obtaining second information provided by the present application;

[0047] Fig.10A flowchart of an implementation of a method for determining first driving area information according to first information and second information provided in the present application;

[0048] Fig.11 A flowchart of another implementation of the method for determining first driving area information according to first information and second information provided by the present application;

[0049] Fig.12 A structural block diagram of an implementation of a device for determining driving area information provided by the present application;

[0050] Fig.13 A structural block diagram of another embodiment of the device for determining driving area information provided in the present application. DETAILED DESCRIPTION

[0051] The technical solution of the present application is described below in conjunction with the accompanying drawings.

[0052] First, the system for determining the driving area information provided by the present application is described. The method for determining the driving area information provided by the present application can be implemented in the system.

[0053] See also Figure 1 and Figure 2 , Figure 1 A structural block diagram of an implementation of a system for determining driving area information provided in the present application. Figure 2 This is a structural block diagram of another embodiment of the system for determining driving area information provided by the present application. Figure 1 and Figure 2 It can be seen that the system 100 may include: a camera module 10, a radar 20, and a fusion module 30; wherein the camera module 10 includes at least one, and the camera module 10 can be used to capture images of the driving environment in which the vehicle is located, and determine the information of the initial driving area based on the captured images; the radar 20 can be a laser radar or a vehicle-mounted radar, which can be used to emit lasers or electromagnetic waves to the driving environment in which the vehicle is located, and generate radar detection information based on the echo of the laser or electromagnetic waves; the fusion module 30 can be used to fuse the information of the initial driving area determined by the camera module 10 and the radar detection information generated by the radar 20 to obtain the driving area information of the vehicle, so that the vehicle is safer when driving according to the driving area information. Among them, the fusion module 30 can be set separately (such as Figure 1 ), or may be disposed in the camera module 10 (as shown Figure 2 shown).

[0054] Those skilled in the art will understand that Figure 1 and Figure 2The structure of the system for determining the driving area information shown does not constitute a limitation on the system for determining the driving area information of the present application. The system for determining the driving area information of the present application may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0055] The following describes an embodiment of the method for determining driving area information provided by the present application.

[0056] See also Figure 3 , Figure 3 This is a flow chart of an implementation method of the method for determining driving area information provided by the present application. The execution subject of the method can be a separately set fusion module, or a camera module with a fusion module set inside, combined with Figure 3 It can be seen that the method includes:

[0057] Step 101: Obtain first information.

[0058] The first information includes information of an initial driving area determined according to at least one image, and the at least one image comes from at least one camera module. Further, the specific content of the first information can refer to the content of the subsequent embodiments.

[0059] There are many ways to obtain the first information. Figure 3 When the execution subject of the method shown is a camera module with a fusion module installed inside, such as Figure 4 As shown, obtaining the first information can be achieved by following the steps below:

[0060] Step 201: The camera module captures an image of the area to be detected.

[0061] Among them, the area to be detected refers to the detection area in the driving environment where the vehicle is located, and the driving area information is to be determined. That is to say, in the driving environment where the vehicle is located, the driving area information in which detection areas needs to be determined can be determined as the area to be detected. For example, when it is necessary to determine the driving area information in the detection area directly in front of the vehicle, the detection area directly in front of the vehicle can be determined as the area to be detected; or, when it is necessary to determine the driving area information in the detection area behind the vehicle, the detection area behind the vehicle can be determined as the area to be detected; and so on, when it is necessary to determine the driving area information in other detection areas in the driving environment where the vehicle is located, the detection area can be determined as the area to be detected, and they are not listed one by one here.

[0062] The image of the area to be detected may be one or more images. If there are more images of the area to be detected, the images may be obtained by taking pictures with one camera module or with more than one camera modules.

[0063] Step 202: The camera module performs classification and recognition processing on the image to obtain image information of the area to be detected.

[0064] Optionally, the camera module may use a CNN-based semantic segmentation algorithm or an instance segmentation algorithm to classify and identify the image to obtain image information of the area to be detected. The image information of the area to be detected includes information of m pixels corresponding to the area to be detected, and the information of the m pixels can be expressed in a pixel coordinate system as [(a1, b1, c1), (a2, b2, c2), ..., (a i ,b i ,c i ),…,(a m ,b m ,c m )]; wherein said a i ,b i represents the position coordinates of the i-th pixel point corresponding to the area to be detected, and the c i represents the category of the i-th pixel; m is an integer greater than or equal to 1.

[0065] It should be noted that the image information of the area to be detected can also be converted into a vehicle body coordinate system or a world coordinate system.

[0066] Step 203: The camera module determines first information according to the image information, where the first information includes information of an initial driving area.

[0067] After the camera module obtains the image information of the area to be detected, the information of all pixels of the drivable category is determined as the information of the initial driving area according to the category corresponding to each pixel in the image information. Correspondingly, the area composed of all pixels of the drivable category is determined as the initial driving area.

[0068] The information of the initial driving area can be expressed in various ways, for example:

[0069] The first method: the information of the initial driving area includes information of n pixel points corresponding to the initial driving area. The information of the n pixel points can be expressed as [(d1, e1), (d2, e2), ..., (d i ,e i ),…,(d n ,e n )]; wherein, the d i ,e i represents the position coordinates of the i-th pixel point corresponding to the initial driving area; n is an integer greater than or equal to 1, and n is less than or equal to m.

[0070] The second method: the information of the initial driving area can be represented by the boundary information of the initial driving area. The boundary information includes information of f pixels, and the information of the f pixels can be represented in the pixel coordinate system as [(u1,h1),(u2,h2),(u3,h3),…,(u i ,h i ),…,(u f ,h f )]; wherein, the u i ,h i represents the position coordinates of the i-th pixel point in the boundary information; f is an integer greater than or equal to 1, and f is less than or equal to n.

[0071] The third method: The information of the initial driving area can be represented by the boundary information of the initial driving area. The boundary information includes information of r pixels, and the information of the r pixels can be represented in the pixel coordinate system as [step, (g1, k1), k2, k3, ..., k i , …, k r ] or [step,(g1,k1),k2,k3,…,k i , …, k r , method]; wherein, g1, k1 represents the position coordinates of the first pixel in the boundary information; i represents the ordinate of the ith pixel in the boundary information, and the abscissa of the ith pixel is obtained according to the step; the step represents the interval step between the abscissas of the ith pixel and the i+1th pixel, and the interval step between the abscissas of the ith pixel and the i-1th pixel is the same as the interval step between the abscissas of the ith pixel and the i+1th pixel; the method represents the connection relationship between all interval steps; the r is an integer greater than or equal to 1, and the r is less than the n.

[0072] Fourth method: The information of the initial driving area can be represented by the boundary information of the initial driving area. The boundary information includes information of s pixels, and the information of the s pixels can be represented in the pixel coordinate system as [(p1, q1), (p2, q2), (p3, q3), ..., (p i ,q i ),…,(p s ,q s ),method]; wherein, the p i ,q iRepresents the position coordinates of the i-th pixel point in the boundary information; there is an interval step between the i-th pixel point and the i+1-th pixel point abscissa, and the interval step between the i-th pixel point and the i-1-th pixel point abscissa is different from the interval step between the i-th pixel point and the i+1-th pixel point abscissa; the method represents the connection relationship between all interval steps; the s is an integer greater than or equal to 1, and the s is less than the n.

[0073] The method in the third and fourth methods above can be set to multiple methods. For example, the method can be set to linear interpolation, quadratic interpolation, cubic spline interpolation, or shape-preserving piecewise cubic interpolation.

[0074] It should be noted that the information of the initial driving area can also be converted into a vehicle body coordinate system or a world coordinate system.

[0075] use Figure 4 The method shown obtains the first information. Since the fusion module is arranged inside the camera module, the first information is directly obtained through the fusion module, which reduces the amount of data transmission and can subsequently improve the efficiency of determining the driving area information.

[0076] when Figure 3 When the execution subject of the method shown is a separately set fusion module, such as Figure 5 As shown, obtaining the first information can also be achieved by following the steps below:

[0077] Step 301: The camera module captures an image of the area to be detected.

[0078] Step 302: The camera module performs classification and recognition processing on the image to obtain image information of the area to be detected.

[0079] For specific implementations of step 301 and step 302, please refer to Figure 4 Step 201 and step 202 in the illustrated embodiment will not be described in detail here.

[0080] Step 303: The camera module sends the image information to the fusion module.

[0081] Step 304: The fusion module receives the image information sent by the camera module.

[0082] Step 305: The fusion module determines first information according to the image information, where the first information includes information of an initial driving area.

[0083] For the specific implementation of step 305, please refer to Figure 4 Step 203 in the illustrated embodiment will not be described in detail here.

[0084] use Figure 5 The method shown for obtaining the first information can reduce the amount of data calculation in the camera module, reduce the overhead of the camera module, and thus save resources of the camera module.

[0085] when Figure 3 When the execution subject of the method shown is a separately set fusion module, such as Figure 6 As shown, obtaining the first information can also be achieved by following the steps below:

[0086] Step 401: The camera module captures an image of the area to be detected.

[0087] Step 402: The camera module performs classification and recognition processing on the image to obtain image information of the area to be detected.

[0088] Step 403: The camera module determines first information according to the image information, where the first information includes information of an initial driving area.

[0089] For specific implementations of steps 401 to 403, please refer to Figure 4 Steps 201 to 203 in the illustrated embodiment will not be described in detail herein.

[0090] Step 404: The camera module sends the first information to the fusion module.

[0091] When the camera module sends the first information to the fusion module, if the information of the initial driving area included in the first information is expressed according to the second to fourth methods mentioned above, the amount of data transmission can be reduced, and the efficiency of determining the driving area information can be improved subsequently.

[0092] Step 405: The fusion module receives the first information sent by the camera module.

[0093] use Figure 6 The method shown in the figure obtains the first information, which can reduce the amount of data calculation in the fusion module, reduce the overhead of the fusion module, and thus save resources of the fusion module.

[0094] Step 102: Obtain second information.

[0095] The second information includes radar detection information. Further, the specific content of the second information can refer to the content of the subsequent embodiments.

[0096] There are many ways to obtain the second information, such as Figure 7 As shown, obtaining the second information can be achieved by following the steps below:

[0097] Step 501: The radar detects the area to be detected to obtain second information, where the second information includes radar detection information of the area to be detected.

[0098] The radar may be a laser radar or a vehicle-mounted radar. After the radar emits laser or electromagnetic waves to the area to be detected, radar detection information of the area to be detected can be obtained through the echo of the laser or electromagnetic waves.

[0099] The radar detection information of the area to be detected includes information of t target points corresponding to the area to be detected. The information of the t target points can be expressed as [(X1, Y1), (X2, Y2), ..., (X i ,Y i ),…,(X t ,Y t )] or [(X1,Y1,R1),(X2,Y2,R2),…,(X i ,Y i ,R i ),…,(X t ,Y t R t )]; wherein, the X i ,Y i represents the position coordinate information of the i-th target point corresponding to the area to be detected; R i Represents the covariance information of the position coordinates of the i-th target point; t is an integer greater than or equal to 1.

[0100] Step 502: The radar sends the second information to a driving area information acquisition device.

[0101] Step 503: The driving area information acquisition device receives the second information sent by the radar.

[0102] when Figure 3 When the execution subject of the method shown is a separately set fusion module, Figure 7 In the illustrated embodiment, the driving area information acquisition device is a separately provided fusion module. Figure 3 When the execution subject of the method shown is a camera module having a fusion module disposed therein, Figure 7 In the illustrated embodiment, the driving area information acquisition device is a camera module with a fusion module disposed therein.

[0103] use Figure 7 The method shown in the figure obtains the second information, which can reduce the amount of radar calculation and the radar overhead, thereby saving radar resources.

[0104] according to Figure 7In the embodiment shown in the figure, the radar needs to send the radar detection information of the area to be detected to the driving area information acquisition device, which results in a large amount of data transmission. In addition, when the driving area information is subsequently determined based on the first information and the second information, a large amount of data needs to be fused and processed, which results in a large amount of calculation, resulting in a low efficiency in determining the driving area information. Figure 8 As shown, obtaining the second information can also be achieved by following the steps below:

[0105] Step 601: The camera module captures an image of the area to be detected.

[0106] Step 602: The camera module performs classification and recognition processing on the image to obtain image information of the area to be detected.

[0107] Step 603: The camera module determines first information according to the image information, where the first information includes information of an initial driving area.

[0108] For specific implementations of steps 601 to 603, please refer to Figure 4 Steps 201 to 203 in the illustrated embodiment will not be described in detail herein.

[0109] Step 604: The camera module sends the first information to the radar.

[0110] When the camera module sends the first information including the information of the initial driving area to the radar, the information of the initial driving area must be converted into a vehicle body coordinate system or a world coordinate system.

[0111] Step 605: The radar receives the first information sent by the camera module.

[0112] Step 606: The radar detects the area to be detected to obtain radar detection information of the area to be detected.

[0113] For the specific implementation of step 606, please refer to Figure 7 Step 501 in the illustrated embodiment will not be described in detail here.

[0114] Step 607: The radar determines second information based on the first information, where the second information is the detection information of the initial driving area contained in the radar detection information.

[0115] Step 608: The radar sends the second information to the driving area information acquisition device.

[0116] Step 609: The driving area information acquisition device receives the second information sent by the radar.

[0117] when Figure 3When the execution subject of the method shown is a separately set fusion module, Figure 8 In the illustrated embodiment, the driving area information acquisition device is a separately provided fusion module. Figure 3 When the execution subject of the method shown is a camera module having a fusion module disposed therein, Figure 8 In the illustrated embodiment, the driving area information acquisition device is a camera module with a fusion module disposed therein.

[0118] Figure 8 In the illustrated embodiment, the execution order of step 605 and step 606 can be interchanged, that is, step 605 can be executed first and then step 606, or step 606 can be executed first and then step 605. This application does not impose any restrictions on this.

[0119] use Figure 8 To obtain the second information using the method shown, the radar only needs to send the detection information of the initial driving area to the driving area information acquisition device. The data transmission volume is small, and when the driving area information is subsequently determined based on the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0120] like Fig. 9 As shown, obtaining the second information can also be achieved by following the steps below:

[0121] Step 701: The camera module captures an image of the area to be detected.

[0122] Step 702: The camera module performs classification and recognition processing on the image to obtain image information of the area to be detected.

[0123] Step 703: The camera module determines first information according to the image information, where the first information includes information of an initial driving area.

[0124] For specific implementations of steps 701 to 703, please refer to Figure 4 Steps 201 to 203 in the illustrated embodiment will not be described in detail herein.

[0125] Step 704: The camera module sends the first information to the radar.

[0126] When the camera module sends the information of the initial driving area as the first information to the radar, the information of the initial driving area must be converted into a vehicle body coordinate system or a world coordinate system.

[0127] Step 705: The radar receives the first information sent by the camera module.

[0128] Step 706: The radar detects the initial driving area based on the first information to obtain second information, where the second information includes radar detection information of the initial driving area.

[0129] For the specific implementation of step 706, please refer to Figure 7 Step 501 in the illustrated embodiment will not be described in detail here.

[0130] Step 707: The radar sends the second information to the driving area information acquisition device.

[0131] Step 708: The driving area information acquisition device receives the second information sent by the radar.

[0132] when Figure 3 When the execution subject of the method shown is a separately set fusion module, Fig. 9 In the illustrated embodiment, the driving area information acquisition device is a separately provided fusion module. Figure 3 When the execution subject of the method shown is a camera module having a fusion module disposed therein, Fig. 9 In the illustrated embodiment, the driving area information acquisition device is a camera module with a fusion module disposed therein.

[0133] use Fig. 9 To obtain the second information using the method shown, the radar only needs to send the detection information of the initial driving area to the driving area information acquisition device. The data transmission volume is small, and when the driving area information is subsequently determined based on the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0134] Step 103: Determine first driving area information according to the first information and the second information.

[0135] when Figure 3 When the executor of the method shown is a separately set fusion module or a camera module with a fusion module set inside, determining the first driving area information based on the first information and the second information can be implemented in the following manner: fusing the first information and the second information, and determining the information generated after the fusion processing as the first driving area information.

[0136] In order to further improve the accuracy of the first driving area information, the first driving area information can be determined according to the first information and the second information in the following two ways:

[0137] In the first method, Figure 3 The execution subject of the method shown is a camera module with a fusion module installed inside, such as Fig.10As shown, determining the first driving area information according to the first information and the second information can be implemented according to the following steps:

[0138] Step 801: The fusion module determines whether there is a region of interest based on the first information and the second information.

[0139] The fusion module adopts a statistical fusion algorithm, such as a Bayesian algorithm, to perform statistical fusion processing on the first information and the second information, and determines whether there is an area in the initial driving area where the detection result is suspicious, that is, an area where the detection result is inaccurate, according to the result of the statistical fusion processing. In the embodiment of the present application, the area with suspicious detection results is referred to as a region of interest (ROI).

[0140] Step 802: If the fusion module determines that the region of interest does not exist, execute step 804.

[0141] Step 803: If the fusion module determines that the region of interest exists, execute step 805.

[0142] Step 804: The fusion module fuses the first information and the second information, and determines the information obtained after the fusion processing as the first driving area information.

[0143] Step 805: The fusion module fuses the first information and the second information, and determines the information obtained after the fusion processing as the second driving area information.

[0144] Step 806: The fusion module determines the region information of the region of interest.

[0145] The fusion module may determine the region information of the region of interest according to the result of the statistical fusion processing obtained in step 801 .

[0146] Step 807: The camera module detects the region of interest according to the region information of the region of interest determined by the fusion module to obtain a detection result of the region of interest.

[0147] The camera module classifies and identifies the image of the detection area again according to the area information of the area of ​​interest, and determines whether the category of the pixel points corresponding to the area of ​​interest is a drivable category according to the result of the classification and identification, thereby determining whether the area of ​​interest is an area available for vehicle driving.

[0148] Based on this, the detection result of the region of interest includes: the region of interest is a region where the vehicle can travel; or the region of interest is an obstacle region where the vehicle cannot travel.

[0149] Optionally, when executing step 807, the camera module may use a CNN-based semantic segmentation algorithm or an instance segmentation algorithm to classify and identify the image of the area to be detected; it may also use other simpler classifiers, such as a support vector machine classifier (SVM) or an AdaBoost classifier to classify and identify the image of the area to be detected.

[0150] Step 808: The fusion module determines the first driving area information according to the second driving area information and the detection result of the area of ​​interest.

[0151] If the detection result of the region of interest is that the region of interest is an area where the vehicle can travel, the fusion module determines the second travel area information as the first travel area information. Alternatively, if the detection result of the region of interest is that the region of interest is an obstacle area where the vehicle cannot travel, the fusion module determines the remaining information after removing the region information of the region of interest from the second travel area information as the first travel area information.

[0152] use Fig.10 The method shown determines the first driving area information, and the determined first driving area information is more accurate, so that subsequent vehicles are safer when driving according to the first driving area information.

[0153] In the second method, Figure 3 The execution subject of the method shown is a separately set fusion module, such as Fig.11 As shown, determining the first driving area information according to the first information and the second information can be implemented according to the following steps:

[0154] Step 901: The fusion module determines whether there is a region of interest based on the first information and the second information.

[0155] For the specific implementation of step 901, please refer to Fig.10 Step 801 in the illustrated embodiment will not be described in detail here.

[0156] Step 902: If the fusion module determines that the region of interest does not exist, execute step 904.

[0157] Step 903: If the fusion module determines that the region of interest exists, execute step 905.

[0158] Step 904: The fusion module fuses the first information and the second information, and determines the information obtained after the fusion processing as the first driving area information.

[0159] Step 905: The fusion module fuses the first information and the second information, and determines the information obtained after the fusion processing as the second driving area information.

[0160] Step 906: The fusion module determines the region information of the region of interest.

[0161] The specific implementation of step 906 can refer to Fig.10 Step 806 in the illustrated embodiment will not be described in detail here.

[0162] Step 907: The fusion module sends the area information of the region of interest to the camera module.

[0163] Step 908: The camera module receives the area information of the region of interest sent by the fusion module.

[0164] Step 909: The camera module detects the region of interest according to the region information of the region of interest to obtain a detection result of the region of interest.

[0165] For the specific implementation of step 909, please refer to Fig.10 Step 807 in the illustrated embodiment will not be described in detail here.

[0166] Step 910: The camera module sends the detection result of the region of interest to the fusion module.

[0167] Step 911: The fusion module receives the detection result of the region of interest sent by the camera module.

[0168] Step 912: The fusion module determines the first driving area information according to the second driving area information and the detection result of the area of ​​interest.

[0169] The specific implementation of step 912 can refer to Fig.10 Step 808 in the illustrated embodiment will not be described in detail here.

[0170] use Fig.11 The method shown determines the first driving area information, and the determined first driving area information is more accurate, so that subsequent vehicles are safer when driving according to the first driving area information.

[0171] The method for determining the driving area information provided by the present application first obtains the first information, the first information includes the information of the initial driving area determined according to the image, then obtains the second information including the radar detection information, and finally determines the first driving area information according to the first information and the second information. It can be seen that the first driving area information obtained by the method for determining the driving area information provided by the present application integrates the image information and the radar detection information, and has higher accuracy. Secondly, when the camera device or the radar device has inaccurate detection, such as the camera device is in a shadow or blocked environment, or the laser radar is affected by the climate, the vehicle-mounted radar is affected by the noise signal, etc., the accuracy of the first driving area information can be improved through the detection information of the other party to ensure the safety of vehicle driving.

[0172] The following describes an apparatus embodiment corresponding to the above method embodiment.

[0173] See also Fig.12 , Fig.12 This is a structural block diagram of an implementation of the device for determining driving area information provided by the present application. Fig.12 It can be seen that the device 1200 includes:

[0174] The acquisition module 1201 is used to acquire first information; the first information includes information of an initial driving area determined based on at least one image, and the at least one image comes from at least one camera module; the acquisition module 1201 is also used to acquire second information; the second information includes radar detection information; the determination module 1202 is used to determine the first driving area information based on the first information and the second information.

[0175] The device provided by the present application first obtains the first information, the first information includes the information of the initial driving area determined according to the image, then obtains the second information including the radar detection information, and finally determines the first driving area information according to the first information and the second information. It can be seen that the first driving area information obtained by the device for determining the driving area information provided by the present application integrates the image information and the radar detection information, and has higher accuracy. Secondly, when the camera device or the radar device has inaccurate detection, such as the camera device is in a shadow or blocked environment, or the laser radar is affected by the climate, the vehicle-mounted radar is affected by the noise signal, etc., the accuracy of the first driving area information can be improved through the detection information of the other party to ensure the safety of vehicle driving.

[0176] Optionally, the radar detection information includes detection information of the initial driving area.

[0177] Optionally, the determination module 1202 is further used to determine whether a region of interest exists according to the first information and the second information.

[0178] Optionally, the determination module 1202 is also used to determine the existence of an area of ​​interest; the acquisition module 1201 is also used to obtain second driving area information and area information of the area of ​​interest; the acquisition module 1201 is also used to obtain the detection result of the area of ​​interest based on the area information of the area of ​​interest; the determination module 1202 is also used to determine the first driving area information based on the second driving area information and the detection result of the area of ​​interest.

[0179] Optionally, the information of the initial driving area includes information of at least one pixel point corresponding to the initial driving area; or, the information of the initial driving area is indicated by boundary information of the initial driving area.

[0180] Optionally, the radar detection information includes position coordinate information; or, the radar detection information includes position coordinate information and covariance information of the position coordinates.

[0181] Fig.12 The device 1200 provided in the embodiment shown can be a fusion module provided separately, or can be a camera module provided with a fusion module inside. Figure 3 The method provided by the illustrated embodiment achieves the same beneficial effects.

[0182] See also Fig.13 , Fig.13 This is a result block diagram of another embodiment of the device for determining driving area information provided by the present application. Fig.13 It can be seen that the device 1300 includes:

[0183] The receiving module 1301 is used to receive first information; the first information includes information of an initial driving area determined according to at least one image, and the at least one image comes from at least one camera module;

[0184] A generating module 1302 is used to generate second information; the second information includes radar detection information of the initial driving area generated according to the first information;

[0185] The sending module 1303 is configured to send the second information.

[0186] The device provided by this implementation first receives the first information, then generates the second information including the radar detection information of the initial driving area according to the first information, and finally sends the second information to the driving area information acquisition device. It can be seen that in the device provided by this implementation, the radar only needs to send the radar detection information of the initial driving area to the driving area information acquisition device, and the data transmission volume is small. When the first driving area information is subsequently determined according to the first information and the second information, the amount of data that needs to be fused and processed is small, and the amount of calculation is small, thereby improving the efficiency of determining the driving area information.

[0187] In a specific implementation, the embodiment of the present application further provides a computer storage medium, wherein the computer storage medium may store a program, the program includes instructions, and when the program is executed, the program may include some or all steps of the method for determining driving area information provided by the present application. The computer storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0188] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0189] It should be understood that in each embodiment of the present application, the execution order of each step should be determined by its function and internal logic, and the size of the sequence number of each step does not mean the order of execution and does not limit the implementation process of the embodiment.

[0190] In addition, in the description of the present application, unless otherwise specified, "plurality" refers to two or more than two. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the difference.

[0191] Each part of this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiment of the device for determining driving area information, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0192] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0193] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A method for determining driving area information, characterized in that: include: Acquire first information, the first information including information of an initial driving area determined according to at least one image, the at least one image being from at least one camera module; Acquire second information; the second information includes radar detection information, the radar detection information includes position coordinate information, or the radar detection information includes position coordinate information and covariance information of the position coordinates; First driving area information is determined according to the first information and the second information.

2. The method according to claim 1, characterized in that The radar detection information includes detection information of the initial driving area.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Determine whether a region of interest exists according to the first information and the second information.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine that a region of interest exists; Acquire the second driving area information and the area information of the area of ​​interest; Acquire a detection result of the region of interest according to the region information of the region of interest; The first driving area information is determined according to the second driving area information and the detection result of the area of ​​interest.

5. The method according to any one of claims 1 to 4, characterized in that: The information of the initial driving area includes information of at least one pixel point corresponding to the initial driving area; or, The information of the initial driving area is indicated by boundary information of the initial driving area.

6. The method according to any one of claims 1 to 5, characterized in that: The information of the initial driving area satisfies one or more of the following conditions: The information of the initial driving area includes information of n pixel points corresponding to the initial driving area, and the information of the n pixel points is expressed as [(d1, e1), (d2, e2), ..., (d i ,e i ),…,(d n ,e n )]; wherein, the d i ,e i represents the position coordinates of the i-th pixel point corresponding to the initial driving area, wherein n is an integer greater than or equal to 1 and less than or equal to m, and m is the number of pixels corresponding to the area to be detected included in the image information of the area to be detected; The information of the initial driving area is represented by the boundary information of the initial driving area, and the boundary information includes information of f pixels, and the information of the f pixels is represented in the pixel coordinate system as [(u1,h1),(u2,h2),(u3,h3),…,(u i ,h i ),…,(u f ,h f )]; wherein, the u i ,h i represents the position coordinates of the i-th pixel point in the boundary information, and f is an integer greater than or equal to 1 and less than or equal to n; The information of the initial driving area is represented by the boundary information of the initial driving area, and the boundary information includes information of r pixels, and the information of the r pixels is represented in the pixel coordinate system as [step, (g1, k1), k2, k3, ..., k i ,…,k r ] or [step,(g1,k1),k2,k3,…,k i ,…,k r , method]; wherein, (g1, k1) represents the position coordinates of the first pixel point in the boundary information; i represents the ordinate of the ith pixel in the boundary information, and the abscissa of the ith pixel is obtained according to the step; the step represents the interval step between the abscissas of the ith pixel and the i+1th pixel, and the interval step between the abscissas of the ith pixel and the i-1th pixel is the same as the interval step between the abscissas of the ith pixel and the i+1th pixel; the method represents the connection relationship between all the interval steps; r is an integer greater than or equal to 1 and less than n, and the method is set to linear interpolation, quadratic interpolation, cubic spline interpolation, or shape-preserving piecewise cubic interpolation; The information of the initial driving area is represented by the boundary information of the initial driving area, and the boundary information includes information of s pixel points, and the information of the s pixel points is represented in the pixel coordinate system as [(p1, q1), (p2, q2), (p3, q3), ..., (p i ,q i ),…,(p s ,q s ),method]; wherein, the p i ,q i Represents the position coordinates of the i-th pixel point in the boundary information; there is an interval step between the i-th pixel point and the i+1-th pixel point abscissa, and the interval step between the i-th pixel point and the i-1-th pixel point abscissa is different from the interval step between the i-th pixel point and the i+1-th pixel point abscissa; the method represents the connection relationship between all interval steps; the s is an integer greater than or equal to 1 and less than the n.

7. A method for determining driving area information, characterized in that: include: receiving a first message; The first information includes information of an initial driving area determined according to at least one image, the at least one image being from at least one camera module; generating second information; the second information comprising radar detection information of the initial driving area generated according to the first information; The second information is sent.

8. A device for determining driving area information, characterized in that: include: An acquisition module, configured to acquire first information; the first information includes information of an initial driving area determined according to at least one image, wherein the at least one image comes from at least one camera module; The acquisition module is further used to acquire second information; the second information includes radar detection information, the radar detection information includes position coordinate information, or the radar detection information includes position coordinate information and covariance information of the position coordinates; A determination module is used to determine first driving area information according to the first information and the second information.

9. The device according to claim 8, characterized in that The radar detection information includes detection information of the initial driving area.

10. The device according to any one of claims 8 to 9, characterized in that: The determining module is also used for: Determine whether a region of interest exists according to the first information and the second information.

11. The device according to any one of claims 8 to 10, characterized in that: The determining module is also used for: Determine that a region of interest exists; The acquisition module is also used to acquire the second driving area information and the area information of the area of ​​interest; The acquisition module is also used to acquire the detection result of the region of interest according to the region information of the region of interest; The determination module is further configured to determine the first driving area information according to the second driving area information and the detection result of the area of ​​interest.

12. The device according to any one of claims 8 to 11, characterized in that: The information of the initial driving area includes information of at least one pixel point corresponding to the initial driving area; or, The information of the initial driving area is indicated by boundary information of the initial driving area.

13. The device according to any one of claims 8 to 12, characterized in that: The information of the initial driving area satisfies one or more of the following conditions: The information of the initial driving area includes information of n pixel points corresponding to the initial driving area, and the information of the n pixel points is expressed as [(d1, e1), (d2, e2), ..., (d i ,e i ),…,(d n ,e n )]; wherein, the d i ,e i represents the position coordinates of the i-th pixel point corresponding to the initial driving area, wherein n is an integer greater than or equal to 1 and less than or equal to m, and m is the number of pixels corresponding to the area to be detected included in the image information of the area to be detected; The information of the initial driving area is represented by the boundary information of the initial driving area, and the boundary information includes information of f pixels, and the information of the f pixels is represented in the pixel coordinate system as [(u1,h1),(u2,h2),(u3,h3),…,(u i ,h i ),…,(u f ,h f )]; wherein, the u i ,h i represents the position coordinates of the i-th pixel point in the boundary information, and f is an integer greater than or equal to 1 and less than or equal to n; The information of the initial driving area is represented by the boundary information of the initial driving area, and the boundary information includes information of r pixels, and the information of the r pixels is represented in the pixel coordinate system as [step, (g1, k1), k2, k3, ..., k i ,…,k r ] or [step,(g1,k1),k2,k3,…,k i ,…,k r , method]; wherein, (g1, k1) represents the position coordinates of the first pixel point in the boundary information; i represents the ordinate of the ith pixel in the boundary information, and the abscissa of the ith pixel is obtained according to the step; the step represents the interval step between the abscissas of the ith pixel and the i+1th pixel, and the interval step between the abscissas of the ith pixel and the i-1th pixel is the same as the interval step between the abscissas of the ith pixel and the i+1th pixel; the method represents the connection relationship between all the interval steps; r is an integer greater than or equal to 1 and less than n, and the method is set to linear interpolation, quadratic interpolation, cubic spline interpolation, or shape-preserving piecewise cubic interpolation; The information of the initial driving area is represented by the boundary information of the initial driving area, and the boundary information includes information of s pixel points, and the information of the s pixel points is represented in the pixel coordinate system as [(p1, q1), (p2, q2), (p3, q3), ..., (p i ,q i ),…,(p s ,q s ),method]; wherein, the p i ,q i Represents the position coordinates of the i-th pixel point in the boundary information; there is an interval step between the i-th pixel point and the i+1-th pixel point abscissa, and the interval step between the i-th pixel point and the i-1-th pixel point abscissa is different from the interval step between the i-th pixel point and the i+1-th pixel point abscissa; the method represents the connection relationship between all interval steps; the s is an integer greater than or equal to 1 and less than the n.

14. A device for determining driving area information, characterized in that: include: A receiving module, configured to receive first information; The first information includes information of an initial driving area determined according to at least one image, the at least one image being from at least one camera module; A generating module, configured to generate second information; the second information includes radar detection information of the initial driving area generated according to the first information, the radar detection information includes position coordinate information, or the radar detection information includes position coordinate information and covariance information of the position coordinate; A sending module is used to send the second information.

15. A device, characterized in that: The device comprises a processor, wherein the processor is coupled to a memory, and the processor is configured to read instructions in the memory so that the device executes the method according to any one of claims 1 to 7 according to the instructions.

16. A computer storage medium, characterized in that: The computer storage medium includes instructions, and when the instructions are executed on a computer, the computer is enabled to perform the method according to any one of claims 1 to 7.

17. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images