Neural network training, image detection, driving control method and device

Abstract

The present disclosure provides a neural network training, image detection, driving control method, device, electronic device and storage medium. The neural network training method includes: obtaining a sample image and two-dimensional annotation data of the sample image; the two-dimensional annotation data includes detection frame information of a target vehicle in the sample image, and at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle; based on the sample image and the two-dimensional annotation data, training a target neural network including multiple branch networks; wherein, after the sample image is input into the target neural network, each branch network outputs one of the following information respectively: two-dimensional detection frame information of the target vehicle, and at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, and more specifically, to a neural network training, image detection, driving control method, device, electronic device and storage medium. Background Art

[0002] With the development of technology, autonomous driving functions and assisted driving functions are widely used in vehicles. Among them, it is particularly important to detect vehicles on the road.

[0003] Generally, a camera can be set on the vehicle to capture images through the camera. The moving vehicle is detected based on the captured images to obtain a two-dimensional detection result of the moving vehicle. The two-dimensional detection result of the moving vehicle is input into downstream modules such as a tracking module and a ranging module to obtain three-dimensional information of the moving vehicle. However, since the captured images lack spatial information, the accuracy of the three-dimensional information of the moving vehicle obtained based on the two-dimensional detection results is low. Summary of the invention

[0004] In view of this, the present disclosure at least provides a neural network training, image detection, driving control method, device, electronic device and storage medium.

[0005] In a first aspect, the present disclosure provides a neural network training method, comprising:

[0006] Acquire a sample image and two-dimensional annotation data of the sample image; the two-dimensional annotation data includes detection frame information of a target vehicle in the sample image and at least one attribute information capable of characterizing a three-dimensional position and posture of the target vehicle;

[0007] Based on the sample image and the two-dimensional annotation data, training a target neural network including a plurality of branch networks;

[0008] Among them, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle.

[0009] In the above method, a sample image and two-dimensional annotation data of the sample image are obtained, wherein the two-dimensional annotation data includes detection frame information of the target vehicle and at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle; the target neural network is trained using the sample image and the two-dimensional annotation data. Since the two-dimensional annotation data includes at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle, the trained target neural network can output a prediction to obtain at least one attribute information, thereby enriching the data type for determining the three-dimensional posture information of the target vehicle. Subsequently, the prediction output by the target neural network is used to obtain at least one attribute information and detection frame information, so that the three-dimensional posture information of the target vehicle can be determined more accurately.

[0010] At the same time, since the target neural network includes multiple branch networks, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle. The method of outputting the two-dimensional detection frame information and at least one attribute information in parallel improves the information detection efficiency.

[0011] In a possible implementation manner, the attribute information includes at least one of the following:

[0012] first position information of any dividing point on a dividing line between adjacent visible surfaces of the target vehicle, second position information of a contact point between at least one visible wheel of the target vehicle and the ground, and orientation information of the target vehicle;

[0013] The orientation information includes first orientation information and / or second orientation information, and the second orientation indicated by the second orientation information is included in the first orientation indicated by the first orientation information.

[0014] Here, a variety of attribute information is set to enrich the content of the attribute information, so that the three-dimensional position information of the target vehicle can be determined more accurately based on at least one attribute information later.

[0015] In one possible implementation, when the orientation information includes first orientation information, the first orientation information includes: a first category characterizing that the front of the vehicle and the rear of the vehicle are invisible, a second category characterizing that the rear of the vehicle is visible and the front of the vehicle is invisible, and a third category characterizing that the front of the vehicle is visible and the rear of the vehicle is invisible.

[0016] In one possible implementation, when the orientation information includes second orientation information, the second orientation information includes: a first intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the left side of the vehicle is visible; a second intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the right side of the vehicle is visible; a third intermediate category in which the rear of the vehicle is visible and the front of the vehicle is not visible, and the side of the vehicle is not visible; a fourth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the right side of the vehicle is visible; a fifth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the left side of the vehicle is visible; a sixth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the side of the vehicle is not visible; a seventh intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the right side of the vehicle is visible; and an eighth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the left side of the vehicle is visible.

[0017] By adopting the above method, the orientation of the target vehicle can be more accurately represented by setting the first orientation information and / or the second orientation information.

[0018] In a possible implementation manner, when the attribute information includes the first position information of the demarcation point, acquiring the two-dimensional annotation data of the sample image includes:

[0019] Obtaining coordinate information of the demarcation point in the horizontal direction of the image coordinate system corresponding to the sample image;

[0020] The vertical coordinate information indicated by the detection frame information of the target vehicle is determined as the vertical coordinate information of the dividing point in the image coordinate system corresponding to the sample image.

[0021] By adopting the above method, the coordinate information of the demarcation point in the vertical direction of the image coordinate system corresponding to the sample image can be determined, and the coordinate information of the demarcation point in the horizontal direction can be obtained. Then, when the target neural network determines the first position information of the demarcation point, the coordinate information of the demarcation point in the horizontal direction can be determined without determining the coordinate information of the demarcation point in the vertical direction, thereby reducing the types of regression data and avoiding affecting other regression data (such as detection box information) other than the first position information of the demarcation point when there are many types of regression data, thereby reducing the accuracy of other regression data.

[0022] In a possible implementation manner, when the attribute information includes the second position information of the contact point, acquiring the two-dimensional annotation data of the sample image includes:

[0023] Acquire the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image; determine the coordinate information of the vertical direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image; and / or,

[0024] Obtain the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image; determine the coordinate information of the horizontal direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image.

[0025] By adopting the above method, one coordinate information in the second position information of the contact point can be determined, and another coordinate information in the second position information of the contact point can be obtained, thereby reducing the types of regression data. It can avoid affecting other regression data (such as detection box information, detection box category, etc.) other than the second position information of the contact point when there are many types of regression data, thereby reducing the accuracy of other regression data.

[0026] In a second aspect, the present disclosure provides an image detection method, the method comprising:

[0027] Acquire the image to be detected;

[0028] Inputting the image to be detected into a trained target neural network including a plurality of branch networks, and obtaining two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the plurality of branch networks, and at least one attribute information capable of characterizing the three-dimensional posture of the vehicle;

[0029] Based on the two-dimensional detection frame information of the vehicle and the at least one attribute information, a detection result of the image to be detected is determined.

[0030] In the above method, since the target neural network is obtained based on the neural network training method described in the first aspect, the trained target neural network can more accurately output the two-dimensional detection frame information and at least one attribute information of the vehicle, and then based on the two-dimensional detection frame information and at least one attribute information of the vehicle, the detection result of the image to be detected can be determined more accurately.

[0031] In a possible implementation manner, determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information includes:

[0032] Determine depth information of the vehicle based on the two-dimensional detection frame information of the vehicle and the at least one attribute information.

[0033] In a possible implementation manner, determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information includes:

[0034] Based on the image to be detected, two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle, three-dimensional detection data of the vehicle is determined.

[0035] In a possible implementation manner, when the at least one attribute information includes first position information of any boundary point on a boundary line between adjacent visible surfaces of the vehicle and second position information of a contact point between at least one visible wheel of the vehicle and the ground, determining the three-dimensional detection data of the vehicle based on the image to be detected, the two-dimensional detection frame information of the vehicle included in the image to be detected, and the at least one attribute information corresponding to the vehicle includes:

[0036] Determine two-dimensional compact frame information representing a single plane of the vehicle based on the first position information of the demarcation point, the second position information of the contact point, and the two-dimensional detection frame information of the vehicle;

[0037] Based on the two-dimensional compact frame information and the image to be detected, three-dimensional detection data of the vehicle is determined.

[0038] By adopting the above method, since the spatial information contained in the two-dimensional compact frame information is relatively accurate, the three-dimensional detection data of the vehicle can be determined relatively accurately based on the two-dimensional compact frame information and the image to be detected.

[0039] In a third aspect, the present disclosure provides a driving control method, comprising:

[0040] Acquire road images collected by the driving device during driving;

[0041] Using a target neural network trained by the neural network training method according to any one of the first aspects, detecting the road image to obtain target detection data of a target vehicle included in the road image;

[0042] The travel device is controlled based on the object detection data of the object vehicle included in the road image.

[0043] In a fourth aspect, the present disclosure provides a neural network training device, comprising:

[0044] A first acquisition module is used to acquire a sample image and two-dimensional annotation data of the sample image; the two-dimensional annotation data includes detection frame information of a target vehicle in the sample image and at least one attribute information capable of characterizing a three-dimensional position and posture of the target vehicle;

[0045] A training module, used for training a target neural network including a plurality of branch networks based on the sample image and the two-dimensional annotation data;

[0046] Among them, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle.

[0047] In a fifth aspect, the present disclosure provides an image detection device, comprising:

[0048] A second acquisition module is used to acquire the image to be detected;

[0049] A first generation module is used to input the image to be detected into a trained target neural network including multiple branch networks, and obtain two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the multiple branch networks, and at least one attribute information capable of characterizing the three-dimensional posture of the vehicle;

[0050] A determination module is used to determine the detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information.

[0051] In a sixth aspect, the present disclosure provides a driving control device, comprising:

[0052] A third acquisition module is used to acquire road images collected by the driving device during driving;

[0053] A second generating module, configured to detect the road image using a target neural network trained by the neural network training method according to any one of the first aspects, to obtain target detection data of a target vehicle included in the road image;

[0054] A control module is used to control the travel device based on the target detection data of the target vehicle included in the road image.

[0055] In a seventh aspect, the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the steps of the neural network training method as described in the first aspect or any one of the embodiments above; or to perform the steps of the image detection method as described in the second aspect or any one of the embodiments above; or to perform the steps of the driving control method as described in the third aspect.

[0056] In an eighth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program executes the steps of the neural network training method as described in the first aspect or any one of the embodiments above; or when executed, the computer program executes the steps of the image detection method as described in the second aspect or any one of the embodiments above; or when executed, the computer program executes the steps of the driving control method as described in the third aspect.

[0057] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0059] Figure 1 A schematic diagram of a process flow of a neural network training method provided by an embodiment of the present disclosure is shown;

[0060] Figure 2 A schematic diagram of a sample image in a neural network training method provided by an embodiment of the present disclosure is shown;

[0061] Figure 3 A schematic diagram of a target vehicle in a neural network training method provided by an embodiment of the present disclosure is shown;

[0062] Figure 4 A schematic diagram of a target neural network in a neural network training method provided by an embodiment of the present disclosure is shown;

[0063] Figure 5 A schematic diagram of a flow chart of an image detection method provided by an embodiment of the present disclosure is shown;

[0064] Figure 6 A schematic diagram of a two-dimensional compact frame in an image detection method provided by an embodiment of the present disclosure is shown;

[0065] Figure 7 A schematic flow chart of a driving control method provided by an embodiment of the present disclosure is shown;

[0066] Figure 8A schematic diagram of the architecture of a neural network training device provided by an embodiment of the present disclosure is shown;

[0067] Fig. 9 A schematic diagram of the architecture of an image detection device provided by an embodiment of the present disclosure is shown;

[0068] Fig.10 A schematic diagram of the architecture of a driving control device provided by an embodiment of the present disclosure is shown;

[0069] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown;

[0070] Fig.12 A schematic diagram showing the structure of another electronic device provided by an embodiment of the present disclosure is shown;

[0071] Fig.13 A schematic structural diagram of another electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0073] Generally, a camera can be set on the vehicle to collect images through the camera, and the moving vehicle is detected based on the collected images to obtain a two-dimensional detection result of the moving vehicle, and the obtained two-dimensional detection result of the moving vehicle is input into downstream modules such as a tracking module and a ranging module to obtain three-dimensional information of the moving vehicle. However, since the collected images lack spatial information, the accuracy of the three-dimensional information of the moving vehicle obtained based on the two-dimensional detection result is low. In order to alleviate the above problems, the embodiments of the present disclosure provide a neural network training method.

[0074] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0075] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0076] To facilitate understanding of the embodiments of the present disclosure, a neural network training method, an image detection method, and a driving control method disclosed in the embodiments of the present disclosure are first introduced in detail. The execution subject of the neural network training method, the image detection method, and the driving control method provided in the embodiments of the present disclosure is generally a computer device with a certain computing capability, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the neural network training method, the image detection method, and the driving control method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0077] See also Figure 1 FIG. 1 is a flow chart of a neural network training method provided in an embodiment of the present disclosure, the method comprising S101-S102, wherein:

[0078] S101, obtaining a sample image and two-dimensional annotation data of the sample image; the two-dimensional annotation data includes detection frame information of a target vehicle in the sample image and at least one attribute information capable of characterizing a three-dimensional posture of the target vehicle.

[0079] S102, based on the sample image and the two-dimensional annotation data, training a target neural network including a plurality of branch networks.

[0080] Among them, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle.

[0081] In the above method, a sample image and two-dimensional annotation data of the sample image are obtained, wherein the two-dimensional annotation data includes detection frame information of the target vehicle and at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle; the target neural network is trained using the sample image and the two-dimensional annotation data. Since the two-dimensional annotation data includes at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle, the trained target neural network can output a prediction to obtain at least one attribute information, thereby enriching the data type for determining the three-dimensional posture information of the target vehicle. Subsequently, the prediction output by the target neural network is used to obtain at least one attribute information and detection frame information, so that the three-dimensional posture information of the target vehicle can be determined more accurately.

[0082] At the same time, since the target neural network includes multiple branch networks, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle. The method of outputting the two-dimensional detection frame information and at least one attribute information in parallel improves the information detection efficiency.

[0083] S101-S102 are described in detail below.

[0084] For S101:

[0085] The sample image may be any acquired image containing the target vehicle. The image may be an image containing the target vehicle in any scene, for example, the sample image may include an image of the target vehicle traveling on the road, an image of the target vehicle parked in a parking space, etc. The target vehicle may be any motor vehicle, for example, a truck, a car, a van, etc.

[0086] The two-dimensional annotation data includes the detection frame information of the target vehicle in the sample image, and the detection frame information may include the position information of the detection frame, the size information of the detection frame, the category of the detection frame, etc. For example, the category of the detection frame may include small cars, medium-sized cars, large cars, etc., or the category may also include sedans, vans, off-road vehicles, etc.; or the category may also be a first category belonging to motor vehicles, and a second category that does not belong to motor vehicles.

[0087] Exemplarily, the detection frame information may be the position information of the four vertices of the detection frame, the category of the detection frame; or, the detection frame information may also be the position information of the center point of the detection frame, the size information of the detection frame, the category of the detection frame, etc.

[0088] Among them, at least one attribute information capable of characterizing the three-dimensional position and posture of the target vehicle may include at least one of the following: first position information of any dividing point on the dividing line between adjacent visible surfaces of the target vehicle, second position information of the contact point between at least one visible wheel of the target vehicle and the ground, and orientation information of the target vehicle; wherein the orientation information includes first orientation information and / or second orientation information, and the second orientation indicated by the second orientation information is included in the first orientation indicated by the first orientation information. Here, multiple attribute information is set to enrich the content of the attribute information, so that the three-dimensional position and posture information of the target vehicle can be more accurately determined based on at least one attribute information in the future.

[0089] Any demarcation point of the target vehicle may be any point on the demarcation line between two adjacent visible surfaces of the target vehicle on the sample image. For example, the demarcation line may be a line perpendicular to the ground where the headlights of the target vehicle are located, and the demarcation point may be the intersection of the demarcation line and the detection frame. The visible wheel may be a wheel on one side of the visible surface of the target vehicle included in the sample image; preferably, the number of contact points may be two, i.e., the contact points of two visible wheels with the ground.

[0090] See also Figure 2 A schematic diagram of a sample image in a neural network training method is shown, wherein the sample image includes a dividing point 21, contact points between two wheels and the ground, namely a first contact point 22, and a second contact point 23.

[0091] In an optional implementation, when the attribute information includes the first position information of the demarcation point, in S101, obtaining the two-dimensional annotation data of the sample image may include:

[0092] S1011, obtaining coordinate information of the demarcation point in the horizontal direction of the image coordinate system corresponding to the sample image;

[0093] S1012, determining the vertical coordinate information indicated by the detection frame information of the target vehicle as the vertical coordinate information of the demarcation point in the image coordinate system corresponding to the sample image.

[0094] Here, the first position information of the demarcation point includes horizontal coordinate information and vertical coordinate information, that is, the horizontal coordinate information is the horizontal coordinate information in the image coordinate system corresponding to the sample image, and the vertical coordinate information is the vertical coordinate information in the image coordinate system corresponding to the sample image.

[0095] by Figure 2 The demarcation point included in the example is used to illustrate the Figure 2The detection frame information of the target object included in the sample image may include the position information of the four vertices, namely (x1, y1), (x1, y2), (x2, y1), (x2, y2). Then, the horizontal coordinate information x of the demarcation point 21 in the image coordinate system corresponding to the sample image can be obtained. A , and the vertical coordinate information indicated by the detection frame information of the target vehicle is determined as the vertical coordinate information y2 of the demarcation point in the image coordinate system corresponding to the sample image, that is, the first position information of the demarcation point can be obtained as (x A , y2).

[0096] By adopting the above method, the coordinate information of the demarcation point in the vertical direction of the image coordinate system corresponding to the sample image can be determined, and the coordinate information of the demarcation point in the horizontal direction can be obtained. Then, when the target neural network regresses to determine the first position information of the demarcation point, the coordinate information of the demarcation point in the horizontal direction can be determined without determining the coordinate information of the demarcation point in the vertical direction, thereby reducing the types of regression data and avoiding the impact on other regression data (such as detection box information) other than the first position information of the demarcation point when there are many types of regression data, thereby reducing the accuracy of other regression data.

[0097] In an optional implementation, when the attribute information includes the second position information of the contact point, in S101, obtaining the two-dimensional annotation data of the sample image may include:

[0098] Method 1: obtaining the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image; determining the coordinate information of the vertical direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image;

[0099] Method 2: Obtain the vertical coordinate information of the contact point in the image coordinate system corresponding to the sample image; determine the horizontal coordinate information indicated by the detection frame information of the target vehicle as the horizontal coordinate information of the contact point in the image coordinate system corresponding to the sample image.

[0100] In the first method, the horizontal coordinate information of the contact point can be obtained, and the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image can be determined as the vertical coordinate information of the contact point. In the second method, the vertical coordinate information of the contact point can be obtained, and the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image can be determined as the horizontal coordinate information of the contact point.

[0101] When the number of contact points is two, method one may be selected to determine the second position information of one contact point, and method two may be selected to determine the second position information of another contact point.

[0102] by Figure 2 Taking the contact point included in as an example, the horizontal coordinate information x of the first contact point 22 in the image coordinate system corresponding to the sample image can be obtained. B1 , and the vertical coordinate information indicated by the detection frame information of the target vehicle is determined as the vertical coordinate information y2 of the demarcation point in the image coordinate system corresponding to the sample image, that is, the second position information of the first contact point 22 can be (x B1 , y2).

[0103] For the second contact point 23, the coordinate information y of the second contact point 23 in the vertical direction of the image coordinate system corresponding to the sample image can be obtained. B2 , and the horizontal coordinate information indicated by the detection frame information of the target vehicle is determined as the horizontal coordinate information x1 of the first contact point 22 in the image coordinate system corresponding to the sample image. That is, the second position information of the first contact point 22 can be (x1, y B2 ).

[0104] The selection of x1 or x2 can be determined based on the orientation information of the target vehicle in the sample image; or, it can be determined based on the position of the first contact point in the sample image. For example, if the second contact point is on the right side of the first contact point, x2 is selected; if the first contact point is on the left side of the second contact point, x1 is selected. Alternatively, if the orientation information of the target vehicle is the fourth intermediate category or the seventh intermediate category, x1 is selected; if the orientation information of the target vehicle is the fifth intermediate category or the eighth intermediate category, x2 is selected.

[0105] By adopting the above method, one coordinate information in the second position information of the contact point can be determined, and another coordinate information in the second position information of the contact point can be obtained, thereby reducing the types of regression data. This can avoid affecting other regression data (such as detection box information, detection box category, etc.) other than the second position information of the contact point when there are many types of regression data, thereby reducing the accuracy of other regression data.

[0106] In an optional embodiment, when the orientation information includes first orientation information, the first orientation information includes: a first category characterizing that the front of the vehicle and the rear of the vehicle are invisible, a second category characterizing that the rear of the vehicle is visible and the front of the vehicle is invisible, and a third category characterizing that the front of the vehicle is visible and the rear of the vehicle is invisible.

[0107] In an optional embodiment, when the orientation information includes second orientation information, the second orientation information includes: a first intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the left side of the vehicle is visible; a second intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the right side of the vehicle is visible; a third intermediate category in which the rear of the vehicle is visible and the front of the vehicle is not visible, and the side of the vehicle is not visible; a fourth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the right side of the vehicle is visible; a fifth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the left side of the vehicle is visible; a sixth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the side of the vehicle is not visible; a seventh intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the right side of the vehicle is visible; and an eighth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the left side of the vehicle is visible.

[0108] The second orientation indicated by the second orientation information covers the first orientation indicated by the first orientation information, that is, the first category of the first orientation information may include: the first intermediate category and the second intermediate category in the second orientation information; the second category of the first orientation information may include: the third intermediate category, the fourth intermediate category and the fifth intermediate category in the second orientation information; the third category of the first orientation information may include: the sixth intermediate category, the seventh intermediate category and the eighth intermediate category in the second orientation information.

[0109] Knowable Figure 2 The first orientation information of the target vehicle included in is the third category, and the second orientation information is the seventh intermediate category.

[0110] See also Figure 3 Schematic diagram of a target vehicle in a neural network training method shown in FIG. Figure 3 The first orientation information corresponding to the target vehicle 31 can be the first category, and the second orientation information can be the first intermediate category; the first orientation information corresponding to the target vehicle 32 can be the third category, and the second orientation information can be the eighth intermediate category; the first orientation information corresponding to the target vehicle 33 can be the second category, and the second orientation information can be the third intermediate category; the first orientation information corresponding to the target vehicle 34 can be the second category, and the second orientation information can be the fourth intermediate category.

[0111] For S102:

[0112] A sample image containing two-dimensional annotation data can be input into a target neural network to be trained that includes multiple scoring networks, and the target neural network can be trained for multiple rounds until the accuracy of the trained target neural network is greater than a set accuracy threshold, or until the loss value of the trained target neural network is less than a set loss threshold, thereby obtaining a trained target neural network.

[0113] Among them, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle, that is, multiple branch networks in the trained target neural network are used to output the two-dimensional detection frame information of the vehicle and at least one attribute information that can characterize the three-dimensional posture of the vehicle in parallel for any image to be detected.

[0114] See also Figure 4 A schematic diagram of a target neural network in a neural network training method shown in FIG. Figure 4 It includes a sample image 41, a trunk network 42, and multiple branch networks 43. The multiple branch networks may include a branch network corresponding to the detection box information, a branch network corresponding to the category of the target vehicle, a branch network corresponding to the first position information of the demarcation point, a branch network corresponding to the second position information of the first contact point, a branch network corresponding to the second position information of the second contact point, and a branch network corresponding to the orientation information.

[0115] See also Figure 5 FIG. 5 is a flow chart of an image detection method provided by an embodiment of the present disclosure, the method comprising S501-S503, wherein:

[0116] S501, obtaining an image to be detected;

[0117] S502, inputting the image to be detected into a trained target neural network including multiple branch networks, and obtaining two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the multiple branch networks, and at least one attribute information capable of representing the three-dimensional posture of the vehicle;

[0118] S503: Determine a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and at least one attribute information.

[0119] In the above method, since the target neural network is obtained based on the neural network training method described in the first aspect, the trained target neural network can more accurately output the two-dimensional detection frame information and at least one attribute information of the vehicle, and then based on the two-dimensional detection frame information and at least one attribute information of the vehicle, the detection result of the image to be detected can be determined more accurately.

[0120] For S501 and S502:

[0121] The image to be detected can be any image, and the acquired image to be detected is input into a trained target neural network including multiple branch networks, and the two-dimensional detection frame information of the vehicle included in the image to be detected outputted in parallel by multiple branch networks is obtained, as well as at least one attribute information capable of characterizing the three-dimensional posture of the vehicle. For example, the two-dimensional detection frame information corresponding to the vehicle included in the image to be detected, the first position information of the demarcation point, the second position information of the contact point, the orientation information, etc. can be obtained.

[0122] For S503:

[0123] In an optional implementation, determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and at least one attribute information includes:

[0124] Depth information of the vehicle is determined based on two-dimensional detection frame information of the vehicle and at least one attribute information.

[0125] The depth information of the vehicle may be the distance between the center of the vehicle and the image acquisition device that acquires the image to be detected. Exemplarily, the bird's-eye view corresponding to the vehicle may be determined through a coordinate conversion operation process based on the two-dimensional detection frame information and at least one attribute information of the vehicle, and then the depth information of the vehicle may be determined based on the bird's-eye view; or, a neural network trained for determining depth information may be used to determine the depth information of the vehicle based on the two-dimensional detection frame information and at least one attribute information of the vehicle.

[0126] In an optional implementation, determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and at least one attribute information includes:

[0127] Based on the image to be detected, the two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle, three-dimensional detection data of the vehicle is determined.

[0128] Exemplarily, the image to be detected, the two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle can be input into the trained three-dimensional detection neural network to determine the three-dimensional detection data of the vehicle. The three-dimensional detection data of the vehicle can include the position information of the three-dimensional detection frame of the vehicle, the size information of the three-dimensional detection frame, the category of the three-dimensional detection frame, etc.

[0129] In an optional implementation, when at least one attribute information includes first position information of any boundary point on a boundary line between adjacent visible surfaces of the vehicle and second position information of a contact point between at least one visible wheel of the vehicle and the ground, determining three-dimensional detection data of the vehicle based on the image to be detected, two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle includes:

[0130] Step 1: Determine two-dimensional compact frame information representing a single plane of the vehicle based on the first position information of the demarcation point, the second position information of the contact point, and the two-dimensional detection frame information of the vehicle;

[0131] Step 2: Determine the three-dimensional detection data of the vehicle based on the two-dimensional compact frame information and the image to be detected.

[0132] When at least one attribute information includes a demarcation point, the first position information of the demarcation point corresponding to the vehicle can be used to divide the two-dimensional detection frame of the vehicle into two detection frames, and the detection frame containing the contact point is determined as two-dimensional compact frame information of a single plane representing the vehicle. Figure 6 Schematic diagram of a two-dimensional compact frame in an image detection method shown in FIG. , the detection frame on the right side of the dividing line is the determined two-dimensional compact frame, wherein, Figure 6 21 is the dividing point, and 22 and 23 are two different contact points.

[0133] The two-dimensional compact frame information and the image to be detected can then be input into the trained three-dimensional detection neural network to determine the three-dimensional detection data of the vehicle.

[0134] By adopting the above method, since the spatial information contained in the two-dimensional compact frame information is relatively accurate, the three-dimensional detection data of the vehicle can be determined relatively accurately based on the two-dimensional compact frame information and the image to be detected.

[0135] See also Figure 7 FIG. 1 is a flow chart of a driving control method provided by an embodiment of the present disclosure, the method comprising S701-S703, wherein:

[0136] S701, obtaining a road image collected by the driving device during driving;

[0137] S702, using the target neural network trained by the neural network training method described in the above embodiment to detect the road image, and obtain target detection data of the target vehicle included in the road image;

[0138] S703, controlling the travel device based on the target detection data of the target vehicle included in the road image.

[0139] For example, the driving device may be an autonomous driving vehicle, a vehicle equipped with an Advanced Driving Assistance System (ADAS), or a robot, etc. The road image may be an image collected in real time by the driving device during driving.

[0140] When controlling the traveling device, the traveling device may be controlled to accelerate, decelerate, turn, brake, etc., or a voice prompt message may be played to prompt the driver to control the traveling device to accelerate, decelerate, turn, brake, etc.

[0141] In a specific implementation, the road image can be input into the trained target neural network, the road image can be detected, and the target detection data of the target vehicle included in the road image can be obtained; and then the driving device can be controlled based on the target detection data of the target vehicle. Exemplarily, controlling the driving device based on the target detection data of the target vehicle can include: inputting the target detection data of the target vehicle into the trained three-dimensional detection neural network to obtain three-dimensional detection information of the target object, and controlling the driving device based on the detected three-dimensional detection information of the target object.

[0142] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0143] Based on the same concept, the present disclosure also provides a neural network training device, see Figure 8 As shown, it is a schematic diagram of the architecture of the neural network training device provided by the embodiment of the present disclosure, including a first acquisition module 801 and a training module 802. Specifically:

[0144] The first acquisition module 801 is used to acquire a sample image and two-dimensional annotation data of the sample image; the two-dimensional annotation data includes detection frame information of a target vehicle in the sample image and at least one attribute information capable of characterizing a three-dimensional position and posture of the target vehicle;

[0145] A training module 802 is used to train a target neural network including a plurality of branch networks based on the sample image and the two-dimensional annotation data;

[0146] Among them, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle.

[0147] In a possible implementation manner, the attribute information includes at least one of the following:

[0148] first position information of any dividing point on a dividing line between adjacent visible surfaces of the target vehicle, second position information of a contact point between at least one visible wheel of the target vehicle and the ground, and orientation information of the target vehicle;

[0149] The orientation information includes first orientation information and / or second orientation information, and the second orientation indicated by the second orientation information is included in the first orientation indicated by the first orientation information.

[0150] In one possible implementation, when the orientation information includes first orientation information, the first orientation information includes: a first category characterizing that the front of the vehicle and the rear of the vehicle are invisible, a second category characterizing that the rear of the vehicle is visible and the front of the vehicle is invisible, and a third category characterizing that the front of the vehicle is visible and the rear of the vehicle is invisible.

[0151] In one possible implementation, when the orientation information includes second orientation information, the second orientation information includes: a first intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the left side of the vehicle is visible; a second intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the right side of the vehicle is visible; a third intermediate category in which the rear of the vehicle is visible and the front of the vehicle is not visible, and the side of the vehicle is not visible; a fourth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the right side of the vehicle is visible; a fifth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the left side of the vehicle is visible; a sixth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the side of the vehicle is not visible; a seventh intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the right side of the vehicle is visible; and an eighth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the left side of the vehicle is visible.

[0152] In a possible implementation manner, when the attribute information includes the first position information of the demarcation point, the first acquisition module 801, when acquiring the two-dimensional annotation data of the sample image, is used to:

[0153] Obtaining coordinate information of the demarcation point in the horizontal direction of the image coordinate system corresponding to the sample image;

[0154] The vertical coordinate information indicated by the detection frame information of the target vehicle is determined as the vertical coordinate information of the dividing point in the image coordinate system corresponding to the sample image.

[0155] In a possible implementation manner, when the attribute information includes the second position information of the contact point, the first acquisition module 801, when acquiring the two-dimensional annotation data of the sample image, is used to:

[0156] Acquire the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image; determine the coordinate information of the vertical direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image; and / or,

[0157] Obtain the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image; determine the coordinate information of the horizontal direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image.

[0158] Based on the same concept, the present disclosure also provides an image detection device, see Fig. 9 As shown, it is a schematic diagram of the architecture of the image detection device provided by the embodiment of the present disclosure, including a second acquisition module 901, a first generation module 902, and a determination module 903. Specifically:

[0159] The second acquisition module 901 is used to acquire the image to be detected;

[0160] A first generating module 902 is used to input the image to be detected into a trained target neural network including multiple branch networks, and obtain two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the multiple branch networks, and at least one attribute information capable of characterizing the three-dimensional posture of the vehicle;

[0161] The determination module 903 is used to determine the detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information.

[0162] In a possible implementation manner, the determination module 903, when determining the detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information, is used to:

[0163] Determine depth information of the vehicle based on the two-dimensional detection frame information of the vehicle and the at least one attribute information.

[0164] In a possible implementation manner, the determination module 903, when determining the detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information, is used to:

[0165] Based on the image to be detected, two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle, three-dimensional detection data of the vehicle is determined.

[0166] In a possible implementation manner, when the at least one attribute information includes first position information of any boundary point on a boundary line between adjacent visible surfaces of the vehicle and second position information of a contact point between at least one visible wheel of the vehicle and the ground, the determination module 903, when determining the three-dimensional detection data of the vehicle based on the image to be detected, the two-dimensional detection frame information of the vehicle included in the image to be detected, and the at least one attribute information corresponding to the vehicle, is used to:

[0167] Determine two-dimensional compact frame information representing a single plane of the vehicle based on the first position information of the demarcation point, the second position information of the contact point, and the two-dimensional detection frame information of the vehicle;

[0168] Based on the two-dimensional compact frame information and the image to be detected, three-dimensional detection data of the vehicle is determined.

[0169] Based on the same concept, the present disclosure also provides a driving control device, see Fig.10 As shown, it is a schematic diagram of the architecture of the driving control device provided by the embodiment of the present disclosure, including a third acquisition module 1001, a second generation module 1002, and a control module 1003. Specifically:

[0170] The third acquisition module 1001 is used to acquire the road image collected by the driving device during the driving process;

[0171] The second generating module 1002 is used to detect the road image using the target neural network trained by the neural network training method described in the above embodiment to obtain target detection data of the target vehicle included in the road image;

[0172] The control module 1003 is used to control the travel device based on the target detection data of the target vehicle included in the road image.

[0173] In some embodiments, the functions or templates contained in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0174] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Fig.11As shown, it is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, including a processor 1101, a memory 1102, and a bus 1103. Among them, the memory 1102 is used to store execution instructions, including a memory 11021 and an external memory 11022; the memory 11021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 1101, and the data exchanged with the external memory 11022 such as a hard disk. The processor 1101 exchanges data with the external memory 11022 through the memory 11021. When the electronic device 1100 is running, the processor 1101 communicates with the memory 1102 through the bus 1103, so that the processor 1101 executes the following instructions:

[0175] Acquire a sample image and two-dimensional annotation data of the sample image; the two-dimensional annotation data includes detection frame information of a target vehicle in the sample image and at least one attribute information capable of characterizing a three-dimensional position and posture of the target vehicle;

[0176] Based on the sample image and the two-dimensional annotation data, training a target neural network including a plurality of branch networks;

[0177] Among them, after the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information that can characterize the three-dimensional posture of the target vehicle.

[0178] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Fig.12 As shown, it is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, including a processor 1201, a memory 1202, and a bus 1203. Among them, the memory 1202 is used to store execution instructions, including a memory 12021 and an external memory 12022; the memory 12021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 1201, and the data exchanged with the external memory 12022 such as a hard disk. The processor 1201 exchanges data with the external memory 12022 through the memory 12021. When the electronic device 1200 is running, the processor 1201 communicates with the memory 1202 through the bus 1203, so that the processor 1201 executes the following instructions:

[0179] Acquire the image to be detected;

[0180] Inputting the image to be detected into a trained target neural network including a plurality of branch networks, and obtaining two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the plurality of branch networks, and at least one attribute information capable of characterizing the three-dimensional posture of the vehicle;

[0181] Based on the two-dimensional detection frame information of the vehicle and the at least one attribute information, a detection result of the image to be detected is determined.

[0182] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Fig.13 As shown, it is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, including a processor 1301, a memory 1302, and a bus 1303. Among them, the memory 1302 is used to store execution instructions, including a memory 13021 and an external memory 13022; the memory 13021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 1301, and the data exchanged with the external memory 13022 such as a hard disk. The processor 1301 exchanges data with the external memory 13022 through the memory 13021. When the electronic device 1300 is running, the processor 1301 communicates with the memory 1302 through the bus 1303, so that the processor 1301 executes the following instructions:

[0183] Acquire road images collected by the driving device during driving;

[0184] Using the target neural network trained by the neural network training method described in the above embodiment to detect the road image, and obtain target detection data of the target vehicle included in the road image;

[0185] The travel device is controlled based on the object detection data of the object vehicle included in the road image.

[0186] In addition, the embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the neural network training method, image detection method, and driving control method described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0187] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the neural network training method, image detection method, and driving control method described in the above method embodiments. Please refer to the above method embodiments for details, which will not be repeated here.

[0188] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0189] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0190] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0191] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0192] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0193] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A neural network training method, characterized in that: include: Acquire a sample image and two-dimensional annotation data of the sample image; The two-dimensional annotation data includes detection frame information of the target vehicle in the sample image and at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle; Based on the sample image and the two-dimensional annotation data, training a target neural network including a plurality of branch networks; After the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information capable of characterizing the three-dimensional position and posture of the target vehicle; The attribute information includes at least one of the following: first position information of any boundary point on a boundary line between adjacent visible surfaces of the target vehicle, and second position information of a contact point between at least one visible wheel of the target vehicle and the ground.

2. The method according to claim 1, characterized in that The attribute information also includes: orientation information of the target vehicle; The orientation information includes first orientation information and / or second orientation information, and the second orientation indicated by the second orientation information is included in the first orientation indicated by the first orientation information.

3. The method according to claim 2, characterized in that In the case where the orientation information includes first orientation information, the first orientation information includes: a first category characterizing that the front of the vehicle and the rear of the vehicle are not visible, a second category characterizing that the rear of the vehicle is visible and the front of the vehicle is not visible, and a third category characterizing that the front of the vehicle is visible and the rear of the vehicle is not visible.

4. The method according to claim 2 or 3, characterized in that: In the case where the orientation information includes second orientation information, the second orientation information includes: a first intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the left side of the vehicle is visible; a second intermediate category in which the front of the vehicle and the rear of the vehicle are not visible, and the right side of the vehicle is visible; a third intermediate category in which the rear of the vehicle is visible and the front of the vehicle is not visible, and the side of the vehicle is not visible; a fourth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the right side of the vehicle is visible; a fifth intermediate category in which the rear of the vehicle is visible, the front of the vehicle is not visible, and the left side of the vehicle is visible; a sixth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the side of the vehicle is not visible; a seventh intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the right side of the vehicle is visible; and an eighth intermediate category in which the front of the vehicle is visible, the rear of the vehicle is not visible, and the left side of the vehicle is visible.

5. The method according to claim 2, characterized in that: In a case where the attribute information includes the first position information of the demarcation point, acquiring the two-dimensional annotation data of the sample image includes: Obtaining coordinate information of the demarcation point in the horizontal direction of the image coordinate system corresponding to the sample image; The vertical coordinate information indicated by the detection frame information of the target vehicle is determined as the vertical coordinate information of the dividing point in the image coordinate system corresponding to the sample image.

6. The method according to claim 2, characterized in that In a case where the attribute information includes the second position information of the contact point, acquiring the two-dimensional annotation data of the sample image includes: Acquire the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image; determine the coordinate information of the vertical direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image; and / or, Obtain the coordinate information of the contact point in the vertical direction of the image coordinate system corresponding to the sample image; determine the coordinate information of the horizontal direction indicated by the detection frame information of the target vehicle as the coordinate information of the contact point in the horizontal direction of the image coordinate system corresponding to the sample image.

7. An image detection method, characterized in that: The method comprises: Acquire the image to be detected; Inputting the image to be detected into a trained target neural network including a plurality of branch networks, obtaining two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the plurality of branch networks, and at least one attribute information capable of characterizing the three-dimensional posture of the vehicle; Determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information; The attribute information includes at least one of the following: first position information of any boundary point on a boundary line between adjacent visible surfaces of the target vehicle, and second position information of a contact point between at least one visible wheel of the target vehicle and the ground.

8. The method according to claim 7, characterized in that Determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information includes: Determine depth information of the vehicle based on the two-dimensional detection frame information of the vehicle and the at least one attribute information.

9. The method according to claim 7, characterized in that: Determining a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information includes: Based on the image to be detected, two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle, three-dimensional detection data of the vehicle is determined.

10. The method according to claim 9, characterized in that In a case where the at least one attribute information includes first position information of any boundary point on a boundary line between adjacent visible surfaces of the vehicle and second position information of a contact point between at least one visible wheel of the vehicle and the ground, determining three-dimensional detection data of the vehicle based on the image to be detected, two-dimensional detection frame information of the vehicle included in the image to be detected, and at least one attribute information corresponding to the vehicle includes: Based on the first position information of the demarcation point, the second position information of the contact point and the two-dimensional detection frame information of the vehicle, determine the two-dimensional compact frame information representing the single plane of the vehicle; the two-dimensional compact frame is a detection frame corresponding to the single plane of the vehicle determined from the two-dimensional detection frame of the vehicle; Based on the two-dimensional compact frame information and the image to be detected, three-dimensional detection data of the vehicle is determined.

11. A driving control method, characterized in that: include: Acquire road images collected by the driving device during driving; Detecting the road image using a target neural network trained by the neural network training method according to any one of claims 1 to 6 to obtain target detection data of a target vehicle included in the road image; The travel device is controlled based on the object detection data of the object vehicle included in the road image.

12. A neural network training device, characterized in that: include: A first acquisition module, used to acquire a sample image and two-dimensional annotation data of the sample image; The two-dimensional annotation data includes detection frame information of the target vehicle in the sample image and at least one attribute information capable of characterizing the three-dimensional posture of the target vehicle; A training module, used for training a target neural network including a plurality of branch networks based on the sample image and the two-dimensional annotation data; After the sample image is input into the target neural network, each branch network outputs one of the following information: two-dimensional detection frame information of the target vehicle, and at least one attribute information capable of characterizing the three-dimensional position and posture of the target vehicle; The attribute information includes at least one of the following: first position information of any boundary point on a boundary line between adjacent visible surfaces of the target vehicle, and second position information of a contact point between at least one visible wheel of the target vehicle and the ground.

13. An image detection device, characterized in that: The device comprises: A second acquisition module is used to acquire the image to be detected; A first generation module is used to input the image to be detected into a trained target neural network including multiple branch networks, and obtain two-dimensional detection frame information of the vehicle in the image to be detected outputted in parallel by the multiple branch networks, and at least one attribute information capable of characterizing the three-dimensional posture of the vehicle; A determination module, configured to determine a detection result of the image to be detected based on the two-dimensional detection frame information of the vehicle and the at least one attribute information; The attribute information includes at least one of the following: first position information of any boundary point on a boundary line between adjacent visible surfaces of the target vehicle, and second position information of a contact point between at least one visible wheel of the target vehicle and the ground.

14. A driving control device, characterized in that: include: A third acquisition module is used to acquire road images collected by the driving device during driving; A second generating module, configured to detect the road image using a target neural network trained by the neural network training method according to any one of claims 1 to 6, and obtain target detection data of a target vehicle included in the road image; A control module is used to control the travel device based on the target detection data of the target vehicle included in the road image.

15. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the neural network training method as described in any one of claims 1 to 6 are executed; or the steps of the image detection method as described in any one of claims 7 to 10 are executed; or the steps of the driving control method as described in claim 11 are executed.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the neural network training method as described in any one of claims 1 to 6; or executes the steps of the image detection method as described in any one of claims 7 to 10; or executes the steps of the driving control method as described in claim 11.

Images