A vehicle chassis position positioning method and device, electronic equipment and storage medium

Abstract

The present application relates to a kind of vehicle chassis position positioning method, device, electronic equipment and storage medium, the method comprises: obtaining the first 2D detection frame and the second 2D detection frame corresponding to the vehicle to be detected, based on the first 2D detection frame and the second 2D detection frame, determine the target orientation and target offset of the vehicle to be detected, based on the target orientation and the target offset, determine the bottom edge quadrilateral detection frame corresponding to the vehicle to be detected, to determine the chassis position of the vehicle to be detected according to the bottom edge quadrilateral detection frame. By the method of the present application, based on target orientation and target offset can be accurately positioned to chassis position, and compared with the scheme of 3D detection frame, can speed up data processing speed, facilitate the deployment of edge.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning and positioning, and in particular, relates to a vehicle chassis position positioning method and device, electronic equipment and a storage medium. BACKGROUND

[0002] When a traffic event detection system determines whether a violation event such as occupying an emergency lane, illegal lane changing, and not driving in a specified lane occurs, the vehicle position needs to be accurately positioned. For a truck with a long body, due to the tilt of the imaging angle, the method of positioning the vehicle chassis position based on the bounding box cannot well describe the position of the vehicle.

[0003] In the prior art, the solutions to the above problems are:

[0004] Post-processing logic correction: the center of gravity of the bounding box and the center of the bottom edge cannot well describe the vehicle position, and the intermediate position between the two points is taken by the post-processing logic to approximately represent the vehicle position. The disadvantage of this method is that the error is large and it is not suitable for all cases.

[0005] Using 3D target detection: 3D target detection can output eight fixed points representing a cube, and only four key points of the lower surface of the cube are needed to accurately position the vehicle chassis position. The disadvantage of this solution is that the 3D target detection model generally takes longer to infer and consumes more computing resources compared to the single-stage target detection model, which is not conducive to edge deployment. SUMMARY

[0006] The technical problem to be solved by the present application is to provide a vehicle chassis position positioning method, device, electronic equipment and storage medium, which aims to solve at least one of the above technical problems.

[0007] In a first aspect, the technical solution of the present application to solve the above technical problem is as follows: a vehicle chassis position positioning method, the method comprising:

[0008] obtaining a first 2D detection box and a second 2D detection box corresponding to a vehicle to be detected, the first 2D detection box being a 2D detection box corresponding to the front of the vehicle to be detected perpendicular to the ground, and the second 2D detection box being a 2D detection box corresponding to the rear of the vehicle to be detected perpendicular to the ground;

[0009] determining a target orientation and a target offset of the vehicle to be detected based on the first 2D detection box and the second 2D detection box, the target offset including an x-axis offset and a y-axis offset, the target offset being the length of the 2D detection box, and the target orientation representing the relative position of the first 2D detection box and the second 2D detection box of the vehicle to be detected.

[0010] determine a bottom side quadrilateral detection frame corresponding to the to-be-detected vehicle based on the target orientation and the target offset, so as to determine the chassis position of the to-be-detected vehicle based on the bottom side quadrilateral detection frame.

[0011] The present application has the beneficial effect that: based on the first 2D detection frame and the second 2D detection frame corresponding to the to-be-detected vehicle corresponding to the to-be-detected vehicle, the features representing the chassis position of the to-be-detected vehicle, i.e. the target orientation and the target offset, are determined, so that the chassis position can be accurately positioned based on the target orientation and the target offset, and compared with the scheme of the 3D detection frame, the data processing speed can be accelerated, and the deployment of the edge end is facilitated.

[0012] Based on the above technical solution, the present application can also be improved as follows.

[0013] Further, the above obtaining the first 2D detection frame and the second 2D detection frame corresponding to the to-be-detected vehicle comprises:

[0014] obtaining an image corresponding to the to-be-detected vehicle;

[0015] obtaining the first 2D detection frame and the second 2D detection frame based on the pre-trained yolo target detection model and the image.

[0016] The beneficial effect of the above further scheme is that based on the pre-trained yolo target detection model, the first 2D detection frame and the second 2D detection frame can be accurately and quickly determined.

[0017] Further, the above determining the target orientation and the target offset of the to-be-detected vehicle based on the first 2D detection frame and the second 2D detection frame comprises:

[0018] if the position of the first 2D detection frame is on the left side of the position of the second 2D detection frame, the target orientation of the to-be-detected vehicle is determined to be positive;

[0019] if the position of the first 2D detection frame is on the right side of the position of the second 2D detection frame, the target orientation of the to-be-detected vehicle is determined to be negative;

[0020] determining one side of the first 2D detection frame parallel to the ground as the x-axis offset in the target offset, and determining one side of the second 2D detection frame perpendicular to the ground as the y-axis offset in the target offset.

[0021] The beneficial effect of adopting the above-mentioned further scheme is that since the first 2D detection frame and the second 2D detection frame correspond to the front position and the rear position of the vehicle to be detected respectively, the front position and the rear position can correspond to the chassis position of the vehicle. Therefore, based on the first 2D detection frame and the second 2D detection frame, the target orientation and target offset of the vehicle to be detected can be accurately determined.

[0022] Furthermore, the above-mentioned obtaining of the first 2D detection frame and the second 2D detection frame corresponding to the vehicle to be detected and determining the target orientation and target offset of the vehicle to be detected based on the first 2D detection frame and the second 2D detection frame are determined based on a pre-trained detection model, and the detection model is trained based on the following method:

[0023] Acquire training data, the training data including sample data of each sample vehicle, each sample data including a first identifier and a second identifier corresponding to the sample vehicle, each first identifier representing a true vehicle orientation of the corresponding sample vehicle, and each second identifier representing a true offset of the corresponding sample vehicle;

[0024] Training the initial model based on the training data to obtain a first prediction result and a second prediction result corresponding to each of the sample vehicles, wherein each of the first prediction results is a predicted vehicle orientation of the corresponding sample vehicle, and each of the second prediction results is a predicted offset of the corresponding sample vehicle;

[0025] Determining a total loss value of the initial model according to the first prediction result, the second prediction result, the first identifier, and the second identifier of each of the sample vehicles;

[0026] When the total loss value meets the preset training end condition, the initial model that meets the training end condition is determined as the detection model; when the total loss value does not meet the preset training end condition, the parameters of the initial model are adjusted to retrain the initial model based on the adjusted parameters until the total loss value meets the preset training end condition.

[0027] The beneficial effect of adopting the above further solution is that, based on the above model training process, the target orientation and target offset of the vehicle to be detected can be accurately determined.

[0028] Furthermore, the above-mentioned determining the total loss value of the initial model based on the first prediction result, the second prediction result, the first identifier, and the second identifier of each of the sample vehicles includes:

[0029] Determining a first loss value according to the first prediction result and the first identifier of each of the sample vehicles;

[0030] determine a second loss value according to the second prediction result and the second label of each of the sample vehicles;

[0031] determine a total loss value of the initial model according to the first loss value and the second loss value.

[0032] The beneficial effect of the further scheme is that, in the training process, not only the loss between the real offset and the predicted offset is considered, but also the loss between the real vehicle orientation and the predicted vehicle orientation is considered, further improving the training accuracy of the model.

[0033] Further, for each of the sample vehicles, the sample data of the sample vehicle is obtained based on the following manner:

[0034] obtain a 3D detection box corresponding to the sample vehicle;

[0035] determine a third 2D detection box corresponding to the sample vehicle from the 3D detection box, the third 2D detection box being a 2D detection box corresponding to the whole vehicle of the sample vehicle;

[0036] determine a first 2D detection box and a second 2D detection box corresponding to the sample vehicle based on the 3D detection box;

[0037] determine a real vehicle orientation and a real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle;

[0038] determine the sample data of the sample vehicle based on the real vehicle orientation and the real offset of the sample vehicle.

[0039] The beneficial effect of the further scheme is that, for the sample data of each of the sample vehicles, the real vehicle orientation and the real offset of the sample vehicle can be more accurately described based on the 3D detection box, making the sample data more accurate.

[0040] Further, for each of the sample vehicles, the method further comprises:

[0041] obtain a sample image corresponding to the sample vehicle;

[0042] obtain at least two fourth 2D detection boxes corresponding to the sample vehicle based on the pre-trained yolo target detection model and the sample image, each of the fourth 2D detection boxes being a 2D detection box corresponding to the whole vehicle of the sample vehicle;

[0043] match the third 2D detection box and the at least two fourth 2D detection boxes to obtain a target 2D detection box matched with the third 2D detection box;

[0044] The real vehicle orientation and the real offset amount of the sample vehicle are determined based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle.

[0045] The real vehicle orientation and the real offset amount of the sample vehicle are determined based on the first 2D detection box, the second 2D detection box and the target 2D detection box corresponding to the sample vehicle.

[0046] The beneficial effect of the above further scheme is that the third 2D detection box corresponding to the sample vehicle determined directly from the 3D detection box has errors, that is, cannot accurately express the whole vehicle position of the vehicle, and the at least two fourth 2D detection boxes detected based on the pre-trained yolo target detection model can more accurately express the whole vehicle position of the vehicle, therefore, the target 2D detection box obtained by matching the third 2D detection box and the at least two fourth 2D detection boxes can more accurately express the whole vehicle position of the vehicle, so that the real vehicle orientation and the real offset amount of the sample vehicle determined based on the target 2D detection box are more accurate.

[0047] In a second aspect, the present application also provides a vehicle chassis position positioning device to solve the above technical problems, which comprises:

[0048] The acquisition module is configured to acquire a first 2D detection box and a second 2D detection box corresponding to a vehicle to be detected, the first 2D detection box being a 2D detection box corresponding to a vehicle head of the vehicle to be detected perpendicular to the ground, and the second 2D detection box being a 2D detection box corresponding to a vehicle tail of the vehicle to be detected perpendicular to the ground;

[0049] The processing module is configured to determine a target orientation and a target offset amount of the vehicle to be detected based on the first 2D detection box and the second 2D detection box, the target offset amount comprising an x-axis offset amount and a y-axis offset amount, the target offset amount being a side length of a 2D detection box, and the target orientation representing a relative position of the first 2D detection box and the second 2D detection box of the vehicle to be detected;

[0050] The positioning module is configured to determine a bottom-side quadrilateral detection box corresponding to the vehicle to be detected based on the target orientation and the target offset amount, so as to determine a chassis position of the vehicle to be detected according to the bottom-side quadrilateral detection box.

[0051] In a third aspect, the present application also provides an electronic device to solve the above technical problems, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor implements the vehicle chassis position positioning method of the present application when executing the computer program.

[0052] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the vehicle chassis position positioning method.

[0053] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced.

[0055] Figure 1 A flowchart of a vehicle chassis position positioning method provided by an embodiment of the present application;

[0056] Figure 2 A schematic diagram of a 2D detection frame and an offset provided by an embodiment of the present application;

[0057] Figure 3 A flowchart of a bottom-side quadrilateral detection frame provided by an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a training data determination process provided by an embodiment of the present application;

[0059] Figure 5 A schematic diagram of a model training process provided by an embodiment of the present application;

[0060] Figure 6 A schematic diagram of a vehicle chassis position positioning device provided by an embodiment of the present application;

[0061] Figure 7 A schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0062] The principles and features of the present application are described below, and the examples are only used to explain the present application, and are not used to limit the scope of the present application.

[0063] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0064] The scheme provided by the embodiment of the present application can be applied to any application scenario that needs to determine the chassis position of a vehicle. The scheme provided by the embodiment of the present application can be executed by any electronic device, such as a terminal device of a user, including at least one of a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, and a smart vehicle-mounted device.

[0065] The embodiment of the present application provides a possible implementation manner, as shown in the flowchart of a vehicle chassis position positioning method, which can be executed by any electronic device, for example, a terminal device, or jointly executed by a terminal device and a server. For the convenience of description, the method provided by the embodiment of the present application will be described below by taking a terminal device as an execution subject, as shown in the flowchart, which can include the following steps: Figure 1 Figure 1

[0066] In step S10, a first 2D detection box and a second 2D detection box corresponding to a to-be-detected vehicle are obtained, the first 2D detection box is a 2D detection box corresponding to the front of the to-be-detected vehicle perpendicular to the ground, and the second 2D detection box is a 2D detection box corresponding to the rear of the to-be-detected vehicle perpendicular to the ground.

[0067] In step S20, based on the first 2D detection box and the second 2D detection box, a target orientation and a target offset of the to-be-detected vehicle are determined, the target offset includes an x-axis offset and a y-axis offset, the target offset is the side length of the 2D detection box, and the target orientation represents the relative position of the first 2D detection box and the second 2D detection box of the to-be-detected vehicle.

[0068] In step S30, based on the target orientation and the target offset, a bottom-side quadrilateral detection box corresponding to the to-be-detected vehicle is determined, so as to determine the chassis position of the to-be-detected vehicle according to the bottom-side quadrilateral detection box.

[0069] Through the method of the present application, on the basis of the first 2D detection box and the second 2D detection box corresponding to the to-be-detected vehicle corresponding to the to-be-detected vehicle, the features that can represent the chassis position of the to-be-detected vehicle, i.e., the target orientation and the target offset, are determined, so as to accurately position the chassis position based on the target orientation and the target offset, and compared with the scheme of the 3D detection box, the data processing speed can be accelerated, and the deployment of the edge end is facilitated.

[0070] The scheme of the present application will be further described below in combination with the following specific embodiment. In the embodiment, the vehicle chassis position positioning method provided can include the following steps:

[0071] ​​Step S10, obtaining a first 2D detection box and a second 2D detection box corresponding to the vehicle to be detected, the first 2D detection box being a 2D detection box corresponding to the front of the vehicle to be detected perpendicular to the ground, and the second 2D detection box being a 2D detection box corresponding to the rear of the vehicle to be detected perpendicular to the ground;

[0072] The vehicle to be detected refers to a vehicle whose chassis position needs to be determined. The vehicle to be detected can be a vehicle in motion or a vehicle stopped at a certain place. In addition, in the prior art, the chassis position of a vehicle with a long length cannot be accurately determined. Therefore, the vehicle to be detected can be a vehicle with a long length, such as a vehicle longer than a set vehicle type, for example, a trailer.

[0073] The first 2D detection box and the second 2D detection box corresponding to the vehicle to be detected are obtained as follows:

[0074] Obtaining an image corresponding to the vehicle to be detected;

[0075] Based on a pre-trained yolo target detection model and the image, the first 2D detection box and the second 2D detection box are obtained.

[0076] It should be noted that the first 2D detection box and the second 2D detection box can be obtained based on other methods in the prior art, such as being determined from a 3D detection box corresponding to the vehicle to be detected, that is, the first 2D detection box and the second 2D detection box are two 2D detection boxes in the 3D detection box. The present scheme does not limit how the first 2D detection box and the second 2D detection box are obtained.

[0077] As an example, referring to the quadrilateral A in Figure 2 , Figure 2 The quadrilateral A is the first 2D detection box, Figure 2 The quadrilateral B is the second 2D detection box.

[0078] Step S20, determining a target orientation and a target offset of the vehicle to be detected based on the first 2D detection box and the second 2D detection box, the target offset including an x-axis offset and a y-axis offset, the target offset being a side length of the 2D detection box, and the target orientation representing a relative position of the first 2D detection box and the second 2D detection box of the vehicle to be detected;

[0079] The different relative positions can be represented based on an orientation identifier, for example, the first 2D detection box is on the left side of the second 2D detection box, and the orientation identifier is positive, otherwise, the first 2D detection box is on the right side of the second 2D detection box, and the orientation identifier is negative.

[0080] The target orientation can also be understood as: representing the offset direction of the vehicle body of the to-be-detected vehicle compared to a reference direction, the reference direction being a standard direction corresponding to the driving direction of the to-be-detected vehicle, and the standard direction being a north-south direction or an east-west direction.

[0081] In the present application, the target offset can be understood as the offset of the to-be-detected vehicle in the image compared to the position corresponding to the true north and true north direction, that is, the offset of the vehicle body of the to-be-detected vehicle in the image compared to the position of the vehicle body in the real scene is described by the target offset.

[0082] As an example, referring to Figure 2 As shown in the figure, Figure 2 The vehicle in the image shows that the vehicle body is inclined, Figure 2 The x-axis offset and the y-axis offset are shown in the figure, and in the present application, the lower left corner of the circumscribed rectangle of the vehicle in the image can be taken as the origin of the xoy coordinate system to construct the xoy coordinate system. The x-axis offset refers to the offset of the vehicle head position of the to-be-detected vehicle compared to the origin on the x-axis, and the y-axis offset refers to the offset of the vehicle tail position of the to-be-detected vehicle compared to the origin on the y-axis.

[0083] In the present application, the x-axis offset and the y-axis offset can be determined based on the side length of the first 2D detection box and the side length of the second 2D detection box, respectively. Therefore, one implementation manner of the above S20 is:

[0084] S201, if the position of the first 2D detection box is on the left side of the position of the second 2D detection box, it is determined that the target orientation of the to-be-detected vehicle is positive, which can be recorded as 1; or, the first 2D detection box is left of the reference direction, and the second 2D detection box is right of the reference direction, then it is determined that the target orientation of the to-be-detected vehicle is positive.

[0085] S202, if the position of the first 2D detection box is on the right side of the position of the second 2D detection box, it is determined that the target orientation of the to-be-detected vehicle is negative, which can be recorded as -1; or, the first 2D detection box is right of the reference direction, and the second 2D detection box is left of the reference direction, then it is determined that the target orientation of the to-be-detected vehicle is negative.

[0086] S203, one side of the first 2D detection box parallel to the ground is determined as the x-axis offset in the target offset, which can be recorded as offset_x, and one side of the second 2D detection box perpendicular to the ground is determined as the y-axis offset in the target offset, which can be recorded as offset_y.

[0087] In the scheme of the present application, if the first 2D detection box and the second 2D detection box are determined based on the 3D detection box corresponding to the vehicle to be detected, another implementation manner of determining the target offset in step S20 based on the first 2D detection box and the second 2D detection box is:

[0088] S1, determining a circumscribed rectangle corresponding to the vehicle to be detected from the 3D detection box corresponding to the vehicle to be detected, the circumscribed rectangle being a 2D detection box corresponding to the vehicle body of the vehicle to be detected;

[0089] S2, taking the bottom edge of the first 2D detection box corresponding to the vehicle to be detected on the circumscribed rectangle as the x-axis offset in the target offset, the bottom edge being parallel to the ground;

[0090] S3, taking the side edge of the second 2D detection box corresponding to the vehicle to be detected on the circumscribed rectangle as the y-axis offset in the target offset, the side edge being perpendicular to the ground.

[0091] Among them, as an example, the circumscribed rectangle can refer to the quadrilateral C shown in Figure 2

[0092] Since there is an error in the circumscribed rectangle determined from the 3D detection box corresponding to the vehicle to be detected, in the scheme of the present application, it also includes:

[0093] Obtaining an image corresponding to the vehicle to be detected;

[0094] Based on the pre-trained yolo target detection model and the image, obtaining at least two fifth 2D detection boxes corresponding to the sample vehicle, each of the fifth 2D detection boxes being a 2D detection box corresponding to the whole vehicle of the sample vehicle;

[0095] Matching the circumscribed rectangle and the at least two fifth 2D detection boxes to obtain a target detection box matched with the circumscribed rectangle; after determining the target detection box, the circumscribed rectangle in steps S2 and S3 can be replaced by the target detection box.

[0096] Step S30, determining a bottom quadrilateral detection box corresponding to the vehicle to be detected based on the target direction and the target offset, so as to determine the chassis position of the vehicle to be detected according to the bottom quadrilateral detection box.

[0097] ​The implementation of S30 can be: based on the target offset, the size of the bottom quadrilateral bounding box is determined, and based on the target orientation, the offset direction of the bottom quadrilateral bounding box relative to the reference direction is determined, so that the chassis position is determined based on the offset direction and the size of the bottom quadrilateral bounding box, wherein the reference direction is a standard direction corresponding to the driving direction of the vehicle to be detected, and the standard direction is the north-south direction or the east-west direction.

[0098] As an example, see Figure 3 The schematic diagram of the bottom quadrilateral bounding box is shown.

[0099] Optionally, see Figure 4 and Figure 5 S10 and S20 can be determined based on a pre-trained detection model, and the detection model is trained based on the following method:

[0100] S11, obtaining training data (shown in Figure 4 ), the training data includes sample data of each sample vehicle, each sample data includes a first identifier and a second identifier of the corresponding sample vehicle, each first identifier represents the real vehicle orientation of the corresponding sample vehicle, and each second identifier represents the real offset of the corresponding sample vehicle;

[0101] S12, training the initial model according to the training data to obtain the first prediction result and the second prediction result corresponding to each sample vehicle, each first prediction result is the predicted vehicle orientation of the corresponding sample vehicle, and each second prediction result is the predicted offset of the corresponding sample vehicle;

[0102] Wherein, the training data can be sample images, if the training data is used as Figure 5 input data, when the training data is input into the initial model, image features are obtained through feature extraction, and then the first prediction result (corresponding to the vehicle orientation classification toward shown in Figure 5 ) and the second prediction result (corresponding to the x, y axis offset, offset_x, offset_y shown in Figure 5 ) corresponding to each sample vehicle are output based on the image features.

[0103] S13, determining the total loss value of the initial model according to the first prediction result, the second prediction result, the first identifier and the second identifier of each sample vehicle;

[0104] S14, when the total loss value meets the preset training end condition, the initial model that meets the training end condition is determined as the detection model; when the total loss value does not meet the preset training end condition, the parameters of the initial model are adjusted to retrain the initial model based on the adjusted parameters until the total loss value meets the preset training end condition (corresponding to Figure 5 Loss summation and gradient backpropagation as shown in ).

[0105] Optionally, in the above S13, determining the total loss value of the initial model according to the first prediction result, the second prediction result, the first identifier, and the second identifier of each sample vehicle includes:

[0106] S131, determining a first loss value (corresponding to Figure 5 Vehicle orientation classification loss shown in );

[0107] The implementation process of S131 can be implemented based on formula (1):

[0108]

[0109] Where n represents the number of images corresponding to the sample vehicle, which can also be called the number of first prediction results, and w n Represents the loss calculation weight, y n Indicates the nth first identifier, x n Indicates the nth first prediction result, L towards Represents the vehicle orientation classification loss, that is, the first loss value.

[0110] In the present application solution, a binary cross entropy loss function may be used to calculate the first loss value.

[0111] S132, determining a second loss value (corresponding to Figure 5 offset regression loss as shown in );

[0112] Wherein, each second prediction result includes a prediction result corresponding to the x-axis offset and a prediction result corresponding to the y-axis offset, and the second loss value includes a loss value corresponding to the x-axis offset and a loss value corresponding to the y-axis offset;

[0113] The implementation process of the above S132 can be implemented based on formula (2) and formula (3):

[0114]

[0115] Where n represents the number of second prediction results, g n Indicates the nth second identifier, xn the nthsecond prediction result representing the x-axis offset, y n the nthsecond prediction result representing the y-axis offset, L x the loss value corresponding to the x-axis offset, L y the loss value corresponding to the y-axis offset.

[0116] S133, determining a total loss value of the initial model according to the first loss value and the second loss value.

[0117] Optionally, for each of the sample vehicles, the sample data of the sample vehicle further comprises a third identifier, a fourth identifier and a fifth identifier, the third identifier representing a real 2D detection box of the sample vehicle in the corresponding image, the fourth identifier representing a classification result of the sample vehicle, i.e., what type of vehicle it is, and the fifth identifier representing a real probability of the sample vehicle existing in the region of interest.

[0118] After the training data is input into the initial model, a predicted target detection result, a predicted classification result and a predicted probability of the corresponding image of each sample vehicle can also be obtained, the predicted target detection result referring to a predicted 2D detection box of the sample image in the corresponding image, the predicted classification result referring to a category of the sample vehicle, and the predicted probability referring to a probability of the sample vehicle in the region of interest.

[0119] The method can further comprise:

[0120] determining a third loss value (corresponding to a bounding box regression loss shown in FIG. 1) based on each of the third identifiers and each of the predicted target detection results; Figure 5

[0121] determining a fourth loss value (corresponding to a category loss shown in FIG. 1) based on each of the fourth identifiers and each of the predicted classification results; Figure 5

[0122] determining a fifth loss value (corresponding to an objectness loss shown in FIG. 1) based on each of the fifth identifiers and each of the predicted probabilities; Figure 5

[0123] The above S133 can further specifically comprise:

[0124] determining a total loss value of the initial model according to the first loss value, the second loss value, the third loss value, the fourth loss value and the fifth loss value.

[0125] Optionally, for each of the sample vehicles, the sample data of the sample vehicle is acquired based on the following manner:

[0126] acquiring a 3D detection box corresponding to the sample vehicle (corresponding to the 3D detection box shown in FIG. 1) Figure 4 ​​​3D target detection data shown in FIG. 1B) ;

[0127] determining a third 2D detection box corresponding to the sample vehicle from the 3D detection box (corresponding to the bounding box shown in FIG. 1B), the third 2D detection box being a 2D detection box corresponding to the whole vehicle of the sample vehicle; Figure 4

[0128] determining a first 2D detection box and a second 2D detection box corresponding to the sample vehicle based on the 3D detection box; determining the real vehicle orientation and the real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle; Figure 4

[0129] determining the sample data of the sample vehicle based on the real vehicle orientation and the real offset of the sample vehicle.

[0130] Optionally, for each of the sample vehicles, the method further comprises:

[0131] obtaining a sample image corresponding to the sample vehicle;

[0132] obtaining at least two fourth 2D detection boxes (each fourth 2D detection box can also be referred to as a bounding box) corresponding to the sample vehicle based on the pre-trained YOLO target detection model and the sample image, each of the fourth 2D detection boxes being a 2D detection box corresponding to the whole vehicle of the sample vehicle;

[0133] matching the third 2D detection box and the at least two fourth 2D detection boxes to obtain a target 2D detection box matched with the third 2D detection box (corresponding to the matching with the bounding box shown in FIG. 1B) ; Figure 4

[0134] determining the real vehicle orientation and the real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle, comprises:

[0135] determining the real vehicle orientation and the real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the target 2D detection box.

[0136] Optionally, for each of the sample vehicles, determining the real vehicle orientation and the real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle, comprises:

[0137] ​​​determining a real vehicle orientation of the sample vehicle based on the first 2D detection box and the second 2D detection box corresponding to the sample vehicle;

[0138] determining a real offset of the sample vehicle based on the third 2D detection box, the first 2D detection box and the second 2D detection box.

[0139] Optionally, the method for determining the real offset of the sample vehicle based on the third 2D detection box, the first 2D detection box and the second 2D detection box comprises:

[0140] taking a bottom side of the first 2D detection box corresponding to the sample vehicle on the third 2D detection box as an x-axis offset in the real offset;

[0141] taking a side of the second 2D detection box corresponding to the sample vehicle on the third 2D detection box as a y-axis offset in the real offset.

[0142] According to the scheme of the present application, two additional key points (target vehicle orientation and target offset) are output based on the YOLO target detection model to describe the vehicle position. Compared with the scheme of directly using the bounding box to cooperate with the post-processing logic, the vehicle position can be accurately positioned, and end-to-end training can be realized, simplifying the event detection process. Compared with the 3D target detection scheme, the inference speed and the pre-trained base target detection model can be kept consistent while maintaining the chassis positioning accuracy.

[0143] Based on the same principle as the method shown in Figure 1 The present application also provides a vehicle chassis position positioning device 20, which can comprise an acquisition module 210, a processing module 220 and a positioning module 230, as shown in Figure 6

[0144] The acquisition module 210 is configured to acquire a first 2D detection box and a second 2D detection box corresponding to a vehicle to be detected, wherein the first 2D detection box is a 2D detection box corresponding to the front of the vehicle to be detected, and the second 2D detection box is a 2D detection box corresponding to the rear of the vehicle to be detected;

[0145] The processing module 220 is configured to determine a target orientation and a target offset of the vehicle to be detected based on the first 2D detection box and the second 2D detection box, wherein the target offset comprises an x-axis offset and a y-axis offset, the target offset is the length of the 2D detection box, and the target orientation represents the relative position of the first 2D detection box and the second 2D detection box of the vehicle to be detected.​

[0146] determine a bottom side quadrilateral detection frame corresponding to the vehicle to be detected based on the target orientation and the target offset, to determine a chassis position of the vehicle to be detected according to the bottom side quadrilateral detection frame.

[0147] Optionally, the acquisition module 210, when acquiring the first 2D detection frame and the second 2D detection frame corresponding to the vehicle to be detected, is specifically configured to:

[0148] acquire an image corresponding to the vehicle to be detected;

[0149] obtain the first 2D detection frame and the second 2D detection frame based on a pre-trained YOLO target detection model and the image.

[0150] Optionally, the processing module 220, when determining the target orientation and the target offset of the vehicle to be detected based on the first 2D detection frame and the second 2D detection frame, is specifically configured to:

[0151] if the position of the first 2D detection frame is on the left side of the position of the second 2D detection frame, determine that the target orientation of the vehicle to be detected is positive;

[0152] if the position of the first 2D detection frame is on the right side of the position of the second 2D detection frame, determine that the target orientation of the vehicle to be detected is negative;

[0153] determine one side of the first 2D detection frame parallel to the ground as an x-axis offset in the target offset, and determine one side of the second 2D detection frame perpendicular to the ground as a y-axis offset in the target offset.

[0154] Optionally, the processing process of the acquisition module 210 and the processing module 220 is determined based on a pre-trained detection model, and the detection model is obtained based on the following model training module:

[0155] acquire training data, wherein the training data includes sample data of each sample vehicle, each sample data includes a first identifier and a second identifier of a corresponding sample vehicle, each first identifier represents a real vehicle orientation of the corresponding sample vehicle, and each second identifier represents a real offset of the corresponding sample vehicle;

[0156] train an initial model according to the training data to obtain a first prediction result and a second prediction result corresponding to each sample vehicle, each first prediction result is a predicted vehicle orientation of the corresponding sample vehicle, and each second prediction result is a predicted offset of the corresponding sample vehicle;

[0157] determine a total loss value of the initial model according to the first prediction result, the second prediction result, the first label and the second label of each of the sample vehicles;

[0158] when the total loss value meets a preset training end condition, determine an initial model meeting the training end condition as the detection model, and when the total loss value does not meet the preset training end condition, adjust parameters of the initial model, and retrain the initial model based on the adjusted parameters until the total loss value meets the preset training end condition.

[0159] Optionally, when determining the total loss value of the initial model according to the first prediction result, the second prediction result, the first label and the second label of each of the sample vehicles, the model training module is specifically configured to:

[0160] determine a first loss value according to the first prediction result and the first label of each of the sample vehicles;

[0161] determine a second loss value according to the second prediction result and the second label of each of the sample vehicles;

[0162] determine the total loss value of the initial model according to the first loss value and the second loss value.

[0163] Optionally, for each of the sample vehicles, the sample data of the sample vehicle is acquired based on the following manner:

[0164] acquire a 3D detection box corresponding to the sample vehicle;

[0165] determine a third 2D detection box corresponding to the sample vehicle from the 3D detection box, the third 2D detection box being a 2D detection box corresponding to a whole vehicle of the sample vehicle;

[0166] determine a first 2D detection box and a second 2D detection box corresponding to the sample vehicle based on the 3D detection box;

[0167] determine a real vehicle orientation and a real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle;

[0168] determine the sample data of the sample vehicle based on the real vehicle orientation and the real offset of the sample vehicle.

[0169] Optionally, for each of the sample vehicles, the apparatus further comprises:

[0170] The matching module is configured to: acquire a sample image corresponding to the sample vehicle; obtain at least two fourth 2D detection boxes corresponding to the sample vehicle based on a pre-trained YOLO target detection model and the sample image, each of the fourth 2D detection boxes being a 2D detection box corresponding to the whole vehicle of the sample vehicle; and match the third 2D detection box and the at least two fourth 2D detection boxes to obtain a target 2D detection box matched with the third 2D detection box.

[0171] The model training module is specifically configured to determine the real vehicle orientation and the real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle.

[0172] The model training module is specifically configured to determine the real vehicle orientation and the real offset of the sample vehicle based on the first 2D detection box, the second 2D detection box and the third 2D detection box corresponding to the sample vehicle.

[0173] The vehicle chassis position positioning device can execute the vehicle chassis position positioning method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module and unit in the vehicle chassis position positioning device in each embodiment of the present application correspond to the steps in the vehicle chassis position positioning method in each embodiment of the present application. The detailed functions of each module of the vehicle chassis position positioning device can be found in the description of the corresponding vehicle chassis position positioning method provided above, and will not be described here.

[0174] The vehicle chassis position positioning device can be a computer program (including program code) running in a computer device, for example, the vehicle chassis position positioning device is an application software. The device can be used to execute the corresponding steps in the method provided by the embodiments of the present application.

[0175] In some embodiments, the vehicle chassis position positioning apparatus provided by the embodiments of the present application can be implemented in a combination of software and hardware. For example, the vehicle chassis position positioning apparatus provided by the embodiments of the present application can be a hardware decoding processor programmed to execute the vehicle chassis position positioning method provided by the embodiments of the present application. For example, the hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic elements.

[0176] In some other embodiments, the vehicle chassis position positioning apparatus provided by the embodiments of the present application can be implemented in software, Figure 6 The vehicle chassis position positioning apparatus stored in the memory can be software in the form of programs and plug-ins, and includes a series of modules, including an acquisition module 210, a processing module 220, and a positioning module 230, for implementing the vehicle chassis position positioning method provided by the embodiments of the present application.

[0177] The modules described in the embodiments of the present application can be implemented in software or hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0178] Based on the same principles as the method shown in the embodiments of the present application, the embodiments of the present application also provide an electronic device, which can include but is not limited to a processor and a memory; the memory is used to store a computer program; the processor is used to execute the method shown in any of the embodiments of the present application by calling the computer program.

[0179] In an optional embodiment, an electronic device is provided, as shown in Figure 7 Figure 7 ​The electronic device 4000 shown includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that the transceiver 4004 is not limited to one in actual application, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0180] The processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the present disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0181] The bus 4002 can include a path for transmitting information between the above-mentioned components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience, Figure 7 Only one thick line is used in the middle, but it does not mean that there is only one bus or one type of bus.

[0182] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0183] The memory 4003 is configured to store application code (computer program) for implementing the scheme of the present application, and the processor 4001 is configured to control the execution. The processor 4001 is configured to execute the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0184] The electronic device can also be a terminal device, Figure 7 The electronic device shown is only an example, and should not limit the functions and use range of the embodiments of the present application.

[0185] The embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program runs on a computer, the computer can execute the corresponding content in the foregoing method embodiments.

[0186] According to another aspect of the present application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the method provided in the various implementation manners of the above embodiments.

[0187] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0188] It should be understood that the flowchart and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of various embodiments of the present application. In this regard, each block in the flowchart and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0189] The computer readable storage medium of the present application can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0190] The computer readable storage medium described above bears one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to execute the method shown in the above embodiment.

[0191] The above description is merely the preferred embodiments of the present application and the explanation of the applied technical principles. It should be understood by those skilled in the art that the disclosed range of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present application (but not limited to) having similar functions.

Claims

1. A vehicle chassis positioning method, characterized in that: The following steps are involved: Obtain a first 2D detection frame and a second 2D detection frame corresponding to the vehicle to be detected, where the first 2D detection frame is a 2D detection frame corresponding to the front of the vehicle to be detected perpendicular to the ground, and the second 2D detection frame is a 2D detection frame corresponding to the rear of the vehicle to be detected perpendicular to the ground; Determining a target orientation and a target offset of the vehicle to be detected based on the first 2D detection frame and the second 2D detection frame, wherein the target offset includes an x-axis offset and a y-axis offset, and the target offset is the side length of the 2D detection frame. The target orientation represents the relative position of the first 2D detection frame and the second 2D detection frame of the vehicle to be detected; Determining a bottom quadrilateral detection frame corresponding to the vehicle to be detected based on the target orientation and the target offset, so as to determine a chassis position of the vehicle to be detected according to the bottom quadrilateral detection frame; If the first 2D detection frame and the second 2D detection frame are determined based on the 3D detection frame corresponding to the vehicle to be detected, determining the target offset based on the first 2D detection frame and the second 2D detection frame includes: S1, determining a circumscribed rectangle corresponding to the vehicle to be detected from the 3D detection frame corresponding to the vehicle to be detected, where the circumscribed rectangle is a 2D detection frame corresponding to the body of the vehicle to be detected; S2, taking the bottom edge of the first 2D detection frame corresponding to the vehicle to be detected and parallel to the ground on the circumscribed rectangle as the x-axis offset in the target offset; S3, taking the side perpendicular to the ground corresponding to the second 2D detection frame corresponding to the vehicle to be detected on the circumscribed rectangle as the y-axis offset in the target offset; S4, obtaining an image corresponding to the vehicle to be detected; S5: Based on the pre-trained YOLO object detection model and the image, obtain at least two fifth 2D detection frames corresponding to the vehicle to be detected, each of the fifth 2D detection frames being a 2D detection frame corresponding to the entire vehicle to be detected; S6, matching the bounding rectangle with the at least two fifth 2D detection frames to obtain a target detection frame that matches the bounding rectangle; S7, replacing the bounding rectangle in steps S2 and S3 with the target detection frame; The determining, based on the target orientation and the target offset, a bottom quadrilateral detection frame corresponding to the vehicle to be detected, so as to determine a chassis position of the vehicle to be detected according to the bottom quadrilateral detection frame, includes: Based on the target offset, the size of the bottom quadrilateral detection frame is determined, and based on the target orientation, the offset direction of the bottom quadrilateral detection frame relative to the reference direction is determined. Based on the offset direction and the size of the bottom quadrilateral detection frame, the chassis position is determined, wherein the reference direction is the standard direction corresponding to the driving direction of the vehicle to be detected, and the standard direction is the north-south direction or the east-west direction.

2. The method according to claim 1, characterized in that The step of obtaining the first 2D detection frame and the second 2D detection frame corresponding to the vehicle to be detected includes: Acquire an image corresponding to the vehicle to be detected; Based on the pre-trained YOLO object detection model and the image, the first 2D detection box and the second 2D detection box are obtained.

3. The method according to claim 1, characterized in that The determining, based on the first 2D detection frame and the second 2D detection frame, a target orientation and a target offset of the vehicle to be detected includes: If the position of the first 2D detection frame is to the left of the position of the second 2D detection frame, determining that the target orientation of the vehicle to be detected is positive; If the position of the first 2D detection frame is to the right of the position of the second 2D detection frame, determining that the target orientation of the vehicle to be detected is negative; The side of the first 2D detection frame parallel to the ground is determined as the x-axis offset in the target offset, and the side of the second 2D detection frame perpendicular to the ground is determined as the y-axis offset in the target offset.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining of the first 2D detection frame and the second 2D detection frame corresponding to the vehicle to be detected and determining the target orientation and the target offset of the vehicle to be detected based on the first 2D detection frame and the second 2D detection frame are determined based on a pre-trained detection model, and the detection model is trained based on the following method: Acquire training data, the training data including sample data of each sample vehicle, each sample data including a first identifier and a second identifier corresponding to the sample vehicle, each first identifier representing a true vehicle orientation of the corresponding sample vehicle, and each second identifier representing a true offset of the corresponding sample vehicle; Training the initial model based on the training data to obtain a first prediction result and a second prediction result corresponding to each of the sample vehicles, wherein each of the first prediction results is a predicted vehicle orientation of the corresponding sample vehicle, and each of the second prediction results is a predicted offset of the corresponding sample vehicle; Determining a total loss value of the initial model according to the first prediction result, the second prediction result, the first identifier, and the second identifier of each of the sample vehicles; When the total loss value meets the preset training end condition, the initial model that meets the training end condition is determined as the detection model; when the total loss value does not meet the preset training end condition, the parameters of the initial model are adjusted to retrain the initial model based on the adjusted parameters until the total loss value meets the preset training end condition.

5. The method according to claim 4, characterized in that Determining the total loss value of the initial model according to the first prediction result, the second prediction result, the first identifier, and the second identifier of each of the sample vehicles includes: Determining a first loss value according to the first prediction result and the first identifier of each of the sample vehicles; Determine a second loss value according to the second prediction result and the second identifier of each of the sample vehicles; Determine a total loss value of the initial model based on the first loss value and the second loss value.

6. The method according to claim 4, characterized in that For each of the sample vehicles, the sample data of the sample vehicle is obtained based on the following method: Obtaining a 3D detection frame corresponding to the sample vehicle; Determine a third 2D detection frame corresponding to the sample vehicle from the 3D detection frame, where the third 2D detection frame is a 2D detection frame corresponding to the entire sample vehicle; Determining a first 2D detection frame and a second 2D detection frame corresponding to the sample vehicle based on the 3D detection frame; Determining a true vehicle orientation and a true offset of the sample vehicle based on the first 2D detection frame, the second 2D detection frame, and the third 2D detection frame corresponding to the sample vehicle; Based on the actual vehicle orientation and the actual offset of the sample vehicle, sample data of the sample vehicle is determined.

7. The method according to claim 6, characterized in that For each of the sample vehicles, the method further includes: Obtaining a sample image corresponding to the sample vehicle; Based on the pre-trained YOLO object detection model and the sample image, obtaining at least two fourth 2D detection frames corresponding to the sample vehicle, each of the fourth 2D detection frames being a 2D detection frame corresponding to the entire sample vehicle; Matching the third 2D detection frame with the at least two fourth 2D detection frames to obtain a target 2D detection frame that matches the third 2D detection frame; The determining, based on the first 2D detection frame, the second 2D detection frame, and the third 2D detection frame corresponding to the sample vehicle, a true vehicle orientation and a true offset of the sample vehicle includes: Based on the first 2D detection frame, the second 2D detection frame, and the target 2D detection frame corresponding to the sample vehicle, a true vehicle orientation and a true offset of the sample vehicle are determined.

8. A vehicle chassis positioning device, characterized in that: The vehicle chassis positioning method according to claim 1 is adopted, wherein the device comprises: An acquisition module is configured to acquire a first 2D detection frame and a second 2D detection frame corresponding to the vehicle to be detected, wherein the first 2D detection frame is a 2D detection frame corresponding to the front of the vehicle to be detected perpendicular to the ground, and the second 2D detection frame is a 2D detection frame corresponding to the rear of the vehicle to be detected perpendicular to the ground; a processing module, configured to determine a target orientation and a target offset of the vehicle to be detected based on the first 2D detection frame and the second 2D detection frame, wherein the target offset includes an x-axis offset and a y-axis offset, the target offset being the side length of the 2D detection frame, and the target orientation representing the relative position of the first 2D detection frame and the second 2D detection frame of the vehicle to be detected; A positioning module is used to determine a bottom quadrilateral detection frame corresponding to the vehicle to be detected based on the target orientation and the target offset, so as to determine the chassis position of the vehicle to be detected according to the bottom quadrilateral detection frame.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images