Vehicle frame structure design method and device and electronic equipment

Abstract

The embodiment of the invention provides a vehicle frame structure design method and device and electronic equipment. The method comprises the steps that a first view image and a second view image of a first modeling model are acquired; processing the first view image and the second view image through a key point identification model to obtain key point position information of the first view image and key point position information of the second view image; determining three-dimensional information of a plurality of vehicle key points based on the key point position information of the first view image and the key point position information of the second view image; and determining the size of the vehicle frame corresponding to the first modeling model based on the three-dimensional information of the plurality of vehicle key points. According to the technical scheme, the key points of the vehicle are automatically recognized, the size of the vehicle frame is automatically calculated, the efficiency and precision of determining the size of the vehicle frame based on the modeling model can be improved, and rapid model selection of the vehicle body frame in the initial stage of vehicle body design is facilitated.

Description

Technical Field

[0001] This application relates to the field of vehicle design technology, and in particular to a vehicle frame structure design method, apparatus, and electronic device. Background Technology

[0002] The core of concept design lies in obtaining a white body frame model based on the overall vehicle's styling surface, which is a crucial step in the vehicle body structure design process.

[0003] In related technologies, given a styling model, technicians annotate key vehicle points on the model and then calculate the vehicle frame dimensions using traditional geometric algorithms. However, this process of calculating the vehicle frame dimensions based on the styling model is cumbersome and inefficient. Summary of the Invention

[0004] This application provides a vehicle frame structure design method, apparatus, and electronic device, aiming to improve the inefficiency of manually annotating key points of a vehicle and calculating the dimensions of the vehicle frame.

[0005] In a first aspect, embodiments of this application provide a method for determining the dimensions of a vehicle frame. The method includes: acquiring a first view image and a second view image of a first styling model; processing the first view image and the second view image using a pre-trained keypoint recognition model to obtain keypoint location information of the first view image and the second view image; determining three-dimensional information of multiple vehicle key points based on the keypoint location information of the first view image and the second view image; and determining the dimensions of the vehicle frame corresponding to the first styling model based on the three-dimensional information of the multiple vehicle key points.

[0006] By identifying key points of the vehicle in different view images of the first styling model using a key point recognition model, the pixel information of the vehicle key points in the different view images of the first styling model can be used to reconstruct the three-dimensional information of the vehicle key points. The three-dimensional information of multiple vehicle key points can uniquely determine the size of the vehicle frame corresponding to the first styling model. The process of determining the size of the vehicle frame corresponding to the first styling model does not require manual intervention, thus improving the efficiency and accuracy of determining the size of the vehicle frame based on the styling model, which helps to quickly select the body frame in the initial stage of body design.

[0007] Secondly, embodiments of this application provide a vehicle frame size determination device, which includes: an image acquisition module for acquiring a first view image and a second view image of a first styling model; a key point recognition module for processing the first view image using a pre-trained key point recognition model to obtain key point position information of the first view image, and processing the second view image using the key point recognition model to obtain key point position information of the second view image; a three-dimensional information determination module for determining three-dimensional information of multiple vehicle key points based on the key point position information of the first view image and the key point position information of the second view image; and a size determination module for determining the size of the vehicle frame corresponding to the first styling model based on the three-dimensional information of the multiple vehicle key points.

[0008] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to execute the computer program stored in the memory to implement the method described in the first aspect.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method described in the first aspect. Attached Figure Description

[0010] Figure 1 This is a flowchart of a vehicle frame structure design method provided in an embodiment of this application; Figure 2 This is a schematic diagram of key points on a first model provided in an embodiment of this application; Figure 3 This is a flowchart of a vehicle frame structure design method provided in another embodiment of this application; Figure 4 This is a flowchart of a vehicle frame structure design method provided in another embodiment of this application; Figure 5 This is a flowchart of a training key point recognition model provided in an embodiment of this application; Figure 6 yes Figure 5 A schematic diagram of the annotation key points of the first view image and the annotation key points of the second view image involved in the embodiment; Figure 7 This is a structural diagram of the vehicle frame structure design device provided in the embodiments of this application; Figure 8 This is a structural diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0012] Terminology Explanation In this application, the key point recognition model is a model used to identify key points in different view images of a model, and is obtained by training a neural network model with multiple sets of training sample images.

[0013] In this application, the scaling factor refers to the ratio between the actual dimensions of the vehicle frame and the dimensions of the styling model. For example, if the vehicle body length is 4 meters and the length of the vehicle body in the styling model is 1 meter, then the scaling factor is 4 / 1 = 4.

[0014] In this application, the first parameter is the length parameter in the vehicle frame, including but not limited to: front door height, A-pillar head length, A-pillar bottom length, front windshield diagonal length, total length of front and rear door seam, vertical height of front and rear door seam, horizontal length of front door bottom edge, horizontal length of rear door bottom edge, length of rear door wheel arch seam, diagonal length of rear door rear waterline, diagonal length of rear door corner, horizontal length of rear door rear waterline, diagonal length from front door upper seam to rear door corner, total length from C-pillar to rear door rear waterline, C-pillar top length, horizontal length of rear section of roof, total width of sunroof, top width of A-pillar, total length from C-pillar to rear wheel arch, total length from rear wheel arch to rear bumper, etc.

[0015] In this application, the second parameter is an angular parameter in the vehicle frame, including but not limited to: A-pillar tilt angle, the angle between the upper seams of the front and rear doors, the angle between the upper water cut of the rear door and the rear corner, etc.

[0016] This application provides a method for determining the dimensions of a vehicle frame, comprising: acquiring a first view image and a second view image of a first styling model; processing the first view image using a pre-trained keypoint recognition model to obtain keypoint position information of the first view image; and processing the second view image using the keypoint recognition model to obtain keypoint position information of the second view image; determining the three-dimensional coordinates of multiple vehicle keypoints based on the keypoint position information of the first view image and the keypoint position information of the second view image; and determining the dimensions of the vehicle frame corresponding to the first styling model based on the three-dimensional coordinates of the multiple vehicle keypoints.

[0017] Compared to the closest related technologies, the electronic device first acquires a first view image and a second view image of a first styling model. It then uses a key point recognition model to identify the location information of key vehicle points in both the first and second view images. Based on this information, it reconstructs the three-dimensional information of the key vehicle points and finally calculates the dimensions of the vehicle frame. By providing an automated solution for identifying key vehicle points and calculating the dimensions of the vehicle frame without human intervention, the efficiency and accuracy of determining the dimensions of the vehicle frame based on the styling model can be improved, facilitating rapid selection of the vehicle frame in the initial stage of vehicle body design.

[0018] Example 1 This application provides a vehicle frame structure design method. Please refer to... Figure 1 The process includes the following steps.

[0019] S10, Obtain the first view image and the second view image of the first model.

[0020] The first styling model refers to the visible curved surfaces of the vehicle's exterior designed by designers during the body design phase. These include the doors, hood, and headlights, and primarily determine the vehicle's appearance and aerodynamic performance. For example, the first styling model could be a styling model for an SUV.

[0021] The first view image and the second view image are images taken from different perspectives of the first model. The first view image can be any one of the side view, front view, and top view of the first model, and the second view image can be any one of the side view, front view, and top view of the first model. The first view image and the second view image are different. In this application, only an example of a side view for the first view image and a top view for the second view image will be used for illustration.

[0022] The first and second view images of the first model can be directly captured by an electronic device, or captured by another device and then sent to the electronic device.

[0023] S20: The first view image and the second view image are processed by a pre-trained key point recognition model to obtain the key point location information of the first view image and the key point location information of the second view image.

[0024] The key point recognition model is used to identify vehicle key points in the first view image and the second view image, such as... Figure 2As shown, this paper illustrates the numbering and location of key points on a first model provided in an embodiment of this application. The correspondence between the key point numbers and locations is shown in Table-1 below.

[0025] Table 1 The keypoint recognition model is trained on a neural network model using multiple sets of training sample images. Each set of training sample images includes a first view image and a second view image of a vehicle model, and each of the first and second view images is labeled with at least one vehicle keypoint. The neural network model can be a Convolutional Neural Network (CNN), a Feature Pyramid Network (FPN), a Region-based Convolutional Neural Network (R-CNN), a U-net network, etc. In this embodiment, only the U-net network is used as an example for illustration. The training process of the keypoint recognition model will be described in the following embodiments.

[0026] The electronic device inputs the aforementioned first view image and second view image model into the key point recognition model, and the key point recognition model outputs the key point location information of the first view image and the key point location information of the second view image.

[0027] The key point location information of the first view image includes the location information of at least one vehicle key point on the first view image. In some embodiments, the key point location information of the first view image is a heatmap of vehicle key points in the first view image. This heatmap can reflect the probability distribution of whether each pixel in the first view image is a vehicle key point. The darker the color of a pixel in the heatmap, the greater the probability that the pixel at the corresponding position in the first view image is a vehicle key point. In other embodiments, the key point location information of the first view image includes the pixel coordinates of each pixel with a probability greater than a first preset probability. The first preset probability is set based on experiments or experience, and this application embodiment does not limit it.

[0028] The key point location information of the second view image includes the location information of at least one vehicle key point on the second view image. In some embodiments, the key point location information of the second view image is a heatmap of vehicle key points in the second view image. This heatmap can reflect the probability distribution of whether each pixel in the second view image is a vehicle key point. The darker the color of a pixel in the heatmap, the greater the probability that the pixel at the corresponding position in the second view image is a vehicle key point. In other embodiments, the key point location information of the second view image includes the pixel coordinates of each pixel with a probability greater than a second preset probability. The second preset probability is set based on experiments or experience, and this application embodiment does not limit it.

[0029] S30: Based on the key point location information of the first view image and the key point location information of the second view image, determine the three-dimensional information of multiple vehicle key points.

[0030] For the same vehicle key point, its position on the first view image and its position on the second view image should meet the principle of size consistency. Therefore, based on the key point position information of the first view image and the key point position information of the second view image, the three-dimensional information of the vehicle key point can be calculated.

[0031] S40 determines the dimensions of the vehicle frame corresponding to the first styling model based on the three-dimensional information of multiple key vehicle points.

[0032] The dimensions of the vehicle frame include the specific values ​​of various parameters on the vehicle. Optionally, the dimensions of the vehicle frame should include the length value of the length parameter (i.e., the first parameter) and the angle value of the angle parameter (i.e., the second parameter).

[0033] With the three-dimensional information of multiple key vehicle points determined, electronic devices can determine the dimensions of the vehicle frame corresponding to the first styling model based on geometric algorithms.

[0034] The technical solution provided in this application involves an electronic device first acquiring a first view image and a second view image of a styling model. A key point recognition model is then used to identify the location information of key vehicle points in both the first and second view images. Based on this location information, the three-dimensional information of the key vehicle points is reconstructed. Finally, the dimensions of the vehicle frame are calculated based on this three-dimensional information. By providing an automated solution for identifying key vehicle points and calculating the dimensions of the vehicle frame, no manual intervention is required. Therefore, the efficiency and accuracy of determining the dimensions of the vehicle frame based on the styling model can be improved, facilitating rapid selection of the vehicle frame in the initial stage of vehicle body design.

[0035] In some embodiments, S30 includes S310, S320, and S330.

[0036] S310, for each of the multiple vehicle key points, obtain the first coordinates of each vehicle key point in the first coordinate system based on the key point position information of the first view image.

[0037] The first coordinate system is set based on experiments or experience. For ease of calculation, the first coordinate system includes an x-axis and a z-axis. The x-axis is parallel to the length direction of the first model, and the z-axis is parallel to the height direction of the first model. The origin of the first coordinate system is the projection of the feature points on the first model onto the aforementioned placement plane. The aforementioned feature points can be the leftmost point, the rightmost point, the center point, etc. of the model.

[0038] The electronic device first determines the first pixel coordinates of each vehicle key point on the first view image, and then, based on the camera geometry model and the first pixel coordinates, determines the first coordinates of that vehicle key point in the first coordinate system. The first coordinates of the vehicle key point can be determined using... express.

[0039] When the key point location information of the first view image is the key point heatmap of the first view image, since there are usually multiple vehicle key points in the first view image, the electronic device can divide the above key point heatmap into multiple regions, and then, based on the maximum probability solution function of the heatmap, output the pixel coordinates of the pixel that becomes the largest vehicle key point in each region, and obtain the first pixel coordinates of multiple vehicle key points.

[0040] S320: Based on the key point location information of the second view image, obtain the second coordinates of each vehicle key point in the second coordinate system.

[0041] The second coordinate system is set based on experiments or experience, and its origin and one coordinate axis coincide with those of the first coordinate system. Optionally, the second coordinate system includes an x-axis and a y-axis, wherein the y-axis is parallel to the width direction of the first model.

[0042] The electronic device first determines the second pixel coordinates of each vehicle key point on the second view image, and then, based on the camera geometry model and the second pixel coordinates, determines the second coordinates of that vehicle key point in the second coordinate system. The second coordinates of the vehicle key points can be obtained using... express.

[0043] The electronic device divides the key point heatmap of the second view image into multiple regions, and then outputs the pixel coordinates of the pixel that becomes the largest vehicle key point in each region based on the maximum probability solution function of the heatmap, thus obtaining the second pixel coordinates of multiple vehicle key points.

[0044] After acquiring the first coordinates and second coordinates of multiple vehicle key points, the electronic device needs to match them. Optionally, for any vehicle key point's first coordinates, the electronic device acquires the difference between its horizontal coordinate and the horizontal coordinates of other vehicle key points' second coordinates. If the difference is less than a preset difference, it indicates a match. The first and second coordinates are determined when observing the same vehicle key point from both side and top views.

[0045] In other possible implementations, the electronic device determines the first coordinates of each vehicle key point, then determines the second coordinates of that vehicle key point, and then calculates the first and second coordinates of other vehicle key points.

[0046] S330 determines the three-dimensional information of each vehicle's key points based on the first and second coordinates.

[0047] The first coordinate includes the first abscissa and the height coordinate of each vehicle key point in the first coordinate system, and the second coordinate includes the second abscissa and the ordinate of each vehicle key point in the second coordinate system; S330 is specifically implemented as follows: the average of the first abscissa and the second abscissa is determined as the abscissa of each vehicle key point in the three-dimensional coordinate system, the ordinate is determined as the ordinate of each vehicle key point in the three-dimensional coordinate system, and the height coordinate is determined as the height coordinate of each vehicle key point in the three-dimensional coordinate system.

[0048] A three-dimensional coordinate system is obtained by combining the first coordinate system and the second coordinate system.

[0049] For example, the first coordinate of the vehicle's key points in the first coordinate system is used as follows: This indicates that the second coordinate of the vehicle's key points in the second coordinate system is represented by... If the coordinates of the key points of the vehicle in the three-dimensional coordinate system are given, then the coordinates are ( ).

[0050] Since the position of the same vehicle key point in the first view image and its position in the second view image should meet the principle of size consistency, after obtaining the first view image and the second view image, the key point position information of the first view image and the key point position information of the second view image can be identified respectively. Then, based on the key point position information of the first view image, the first coordinate of the vehicle key point in the first coordinate system can be determined, and based on the key point position information of the second view image, the second coordinate of the vehicle key point in the second coordinate system can be determined. By combining the two, the three-dimensional information of the vehicle key point can be accurately obtained.

[0051] Example 2 This application provides another vehicle structure design method, please refer to... Figure 3The process includes the following steps.

[0052] S10, Obtain the first view image and the second view image of the first model.

[0053] S20: The first view image and the second view image are processed by a pre-trained key point recognition model to obtain the key point location information of the first view image and the key point location information of the second view image.

[0054] S30: Based on the key point location information of the first view image and the key point location information of the second view image, determine the three-dimensional information of multiple vehicle key points.

[0055] In some embodiments, the vehicle frame includes a first parameter, which is a length parameter, and S40 includes S410, S420 and S430.

[0056] S410, based on the 3D information of at least two vehicle key points among multiple vehicle key points, determine the model length of the first parameter.

[0057] For any first parameter, the electronic device determines at least two vehicle key points corresponding to the first parameter, and then calculates the distance between the at least two vehicle key points corresponding to the first parameter as the model length of the first parameter.

[0058] For example, the first parameter is the total width of the sunroof, and the y-coordinates of the two key vehicle points corresponding to the total width of the sunroof are respectively... as well as If the x and z coordinates of the two key points of the vehicle are the same, then the total width of the sunroof = .

[0059] For example, the first parameter is the front door height, and the z-coordinates of the two key vehicle points corresponding to the front door height are respectively... as well as If the x and y coordinates of the two key points of the vehicle are the same, then the front door height = .

[0060] For example, the first parameter is the rear door wheel arch gap length, and the x-coordinates of the two key vehicle points corresponding to the rear door wheel arch gap length are respectively... as well as The y-coordinates are respectively as well as If the z-coordinates of the two key points of the vehicle are the same, then the length of the rear door wheel arch gap = .

[0061] For example, the first parameter is the length of pillar A, and the x-coordinates of the two key vehicle points corresponding to the length of pillar A are respectively... as well as The y-coordinates are respectively as well as The z coordinates are respectively as well as Then the length of column A = .

[0062] S420, obtain the scaling factor of the first parameter.

[0063] The scaling factor refers to the ratio between the actual dimensions of the vehicle frame and the dimensions of the first styling model. Optionally, the scaling factors for the vehicle frame and the first styling model are different in different directions. Specifically, the scaling factor for the vehicle frame and the first styling model in the length direction is... ,in, It is the actual length of the vehicle frame. This is the model length of the first styling model; the scaling factor between the vehicle frame and the first styling model in the width direction is... ,in, It is the actual width of the vehicle frame. This is the width of the first styling model; the scaling factor between the vehicle frame and the first styling model in the height direction is... ,in, It is the actual height of the vehicle frame. It is the height of the first model.

[0064] The scaling factor of the first parameter refers to the ratio between the actual length of the first parameter in the vehicle frame and the model length of the first parameter in the first styling model. It can be determined based on at least one of the directional information involved in the first parameter, the scaling factors of the vehicle frame and the first styling model in the length, height, and height directions.

[0065] Optionally, S420 includes S4210-S4230.

[0066] S4210, obtain the three-dimensional directions involved in the first parameter.

[0067] The electronic device can compare the three-dimensional coordinates of two vehicle key points corresponding to the first parameter in a three-dimensional coordinate system. If the three-dimensional coordinates of the two vehicle key points are different at least once, it means that the first parameter involves one three-dimensional direction. If the three-dimensional coordinates of the two vehicle key points are different at least twice, it means that the first parameter involves two three-dimensional directions. If the three-dimensional coordinates of the two vehicle key points are all different, it means that the first parameter involves three three-dimensional directions. The different directions are the directions involved.

[0068] For example, the first parameter is the front door height, and the y-coordinates of the two key vehicle points corresponding to the front door height are respectively... as well as The two are not the same. The x and z coordinates of the two key points of the vehicle are the same, so the height of the front door involves the height direction.

[0069] S4220, when the three-dimensional direction involved in the first parameter is one, the scaling factor corresponding to the three-dimensional direction involved in the first parameter is determined as the scaling factor of the first parameter.

[0070] For example, if the front door height only involves the height direction, then the scaling factor of the vehicle frame and the first styling model in the height direction is the scaling factor of the front door height.

[0071] S4230, when the first parameter involves at least two three-dimensional directions, calculate the scaling factor of the first parameter based on the scaling factors corresponding to the at least two three-dimensional directions involved in the first parameter.

[0072] When the three-dimensional directions involved in the first parameter are the height and length directions, the scaling factor of the first parameter = When the three-dimensional directions involved in the first parameter are the height and width directions, the scaling factor of the first parameter = When the three-dimensional directions involved in the first parameter are the length and width directions, the scaling factor of the first parameter = When the three-dimensional directions involved in the first parameter are length, height, and width, the scaling factor of the first parameter is = .

[0073] S430, based on the scaling factor and model length, determines the actual length of the first parameter.

[0074] The electronic device determines the actual length of the first parameter by multiplying the scaling factor of the first parameter by the model length.

[0075] Table 2 below shows the correspondence between the length parameters in the vehicle frame, the two corresponding vehicle key points, the involved three-dimensional directions, and the dimensional details.

[0076] Table 2 Taking the first row of Table-2 as an example, the length parameter numbered 1 is the front door height, and its size is determined based on the three-dimensional information of the key points numbered 2 and 22 in Table 1. The three-dimensional direction involved is the height direction (z).

[0077] In some embodiments, the vehicle frame includes a second parameter, which is an angle parameter; S40 includes S440 and S450.

[0078] S440, based on the three-dimensional coordinates of at least three vehicle key points among multiple vehicle key points, determines the vectors corresponding to the two sides of the second parameter respectively.

[0079] The vectors corresponding to the two sides of the second parameter are the lines connecting the two key points of the vehicle.

[0080] For example, the second parameter is the inclination angle of column A, and the vectors corresponding to the two sides of the inclination angle of column A are respectively... and ,in, It is the line connecting vehicle key point 3 and vehicle key point 5. ; It is the line connecting vehicle key point 2 and vehicle key point 21. .

[0081] S450, based on the vectors corresponding to the two sides, determines the actual angle of the second parameter.

[0082] Given that the vectors corresponding to the two sides of the second parameter are determined, the electronic device determines the actual angle of the second parameter based on the following dot product formula.

[0083] .

[0084] in, yes The length of the mold, yes The length of the module.

[0085] Table 3 below shows the correspondence between the numbering of the angle parameters in the vehicle frame, the corresponding multiple vehicle key points, the involved three-dimensional directions, and the dimensional details.

[0086] Table 3 Taking the first row of Table-3 as an example, the angle parameter numbered 21 is the tilt angle of the A-pillar. The vector of its first side is determined based on the three-dimensional information of the key points numbered 3 and 5 in Table 1, and the vector of the second side is determined based on the three-dimensional information of the key points numbered 2 and 21 in Table 1. The three-dimensional directions involved are the length direction (x), the width direction (y), and the height direction (z).

[0087] Example 3 This application provides another vehicle frame structure design method, please refer to... Figure 4 The process includes the following steps.

[0088] In this embodiment, S10 includes S110, S120 and S130.

[0089] S110, Obtain the original first view image and the original second view image of the first model.

[0090] Both the original first-view image and the original second-view image are RGB three-channel color images.

[0091] S120, perform grayscale processing on the original first view image to obtain a first grayscale image, and perform grayscale processing on the original second view image to obtain a second grayscale image.

[0092] Grayscale processing converts a three-channel RGB color image into a single-channel grayscale image. In a grayscale image, each pixel value represents brightness, with values ​​ranging from 0 to 255, where a pixel value of 0 represents black and a pixel value of 255 represents white. In this embodiment, the color features of the color image do not provide valuable information; instead, they increase the amount of invalid data, affecting subsequent processing and analysis. Therefore, retaining the three-channel color data is not practically meaningful and would only introduce unnecessary complexity and storage overhead. Thus, the electronic device employs a grayscale processing algorithm to convert the three-channel RGB image into a single-channel grayscale image.

[0093] Grayscale processing algorithms include, but are not limited to, component methods, maximum value methods, average value methods, and weighted average methods. In this embodiment, only the weighted average method is used as an example for grayscale processing. Optionally, the computer device calculates the pixel value of each pixel in the grayscale image using the following formula.

[0094] Gray = k1*R + k2*G + k3*B.

[0095] Where R is the data of the red channel of the pixel in the color image, G is the data of the green channel of the pixel in the color image, B is the data of the blue channel of the pixel in the color image, and k1, k2, and k3 are scaling factors set based on experiments or experience, for example, k1 is 0.2989, k2 is 0.5870, and k3 is 0.1140.

[0096] The grayscale processing algorithms and parameters used for grayscale processing of the original first view image and the original second view image can be the same or different. In this embodiment, only the example where the grayscale processing algorithms and parameters used for grayscale processing of the original first view image and the original second view image are the same is used, which reduces the complexity of grayscale processing.

[0097] By performing grayscale processing on RGB three-channel color images, we can simplify the data structure, reduce storage requirements, improve the efficiency of subsequent processing and analysis, make the algorithm calculate faster, and reduce the waste of computing resources.

[0098] In some embodiments, the first grayscale image and the second grayscale image can be used as input to a keypoint recognition model. That is, the electronic device processes the first grayscale image and the second grayscale image using a pre-trained keypoint recognition model to obtain keypoint location information of the first view image and the second view image.

[0099] In some other embodiments, S10 further includes S130.

[0100] S130, the first grayscale image is downscaled to obtain a first view image, and the second grayscale image is downscaled to obtain a second view image.

[0101] Because grayscale images have high resolution, but the differences between adjacent pixels are small, the feature changes are not obvious enough. Therefore, downscaling is required to enhance the useful information of the image.

[0102] The downscaling methods include, but are not limited to, bilinear interpolation, nearest neighbor interpolation, and bicubic interpolation. In this embodiment, only bilinear interpolation is used as an example for illustrative purposes. Bilinear interpolation calculates the new pixel value by weighted averaging of four neighboring pixels. For example, the electronic device sets the scaling factor involved in the downscaling process to 0.25.

[0103] By downscaling grayscale images, more image details are preserved during the image reduction process. This not only reduces the image size but also makes features more prominent, facilitating subsequent feature extraction and analysis, and improving image processing efficiency and feature recognition accuracy.

[0104] S20: The first view image and the second view image are processed by a pre-trained key point recognition model to obtain the key point location information of the first view image and the key point location information of the second view image.

[0105] S30: Based on the key point location information of the first view image and the key point location information of the second view image, determine the three-dimensional information of multiple vehicle key points.

[0106] S40 determines the dimensions of the vehicle frame corresponding to the first styling model based on the three-dimensional information of multiple key vehicle points.

[0107] Example 4 This application also provides a method for training a key point recognition model. The training process of the key point recognition model and the vehicle frame structure design process provided in Embodiment 1 can be completed by the same electronic device or by different electronic devices. Please refer to... Figure 5 The method includes the following steps.

[0108] S510 acquires multiple sets of training sample images.

[0109] Each set of training sample images includes a first view image and a second view image of the second model, and each of the first and second view images of the vehicle in the second model is labeled with at least one vehicle key point. The second model can be the same as or different from the first model. The number of sets of training sample images can be determined by the accuracy requirements of the key point recognition model. For example, the number of sets of training sample images is 100.

[0110] In the case that the first view image and the second view image in multiple sets of training sample images are RGB three-channel color images, the electronic device for training the key point recognition model will also perform grayscale processing and downscaling processing on the above training sample images. The processed training sample images are used to train the key point recognition model.

[0111] The annotation of the training sample images was done manually. If the same vehicle keypoint was annotated multiple times, a weighted average method could be used to determine the final annotated coordinates.

[0112] Reference Figure 6 This illustrates annotation information for a first view image and annotation information for a second view image provided in an embodiment of this application. Wherein, Figure 6 Part (a) includes the key points marked on the second view image. Figure 6 Part (b) includes the key points marked on the second view image.

[0113] S520: For each set of training sample images in multiple sets of training sample images, input each set of training sample images into the neural network model to obtain the key point location information of the output.

[0114] In this embodiment of the application, the neural network model is a U-net network. The neural network trained by the U-net network can achieve pixel-level key detection, reduce the probability of key points being missed or falsely detected, and thus improve the accuracy of vehicle frame size calculation. In addition, the U-net network can better adapt to small sample tasks, and can train a key point recognition model with higher accuracy even with a small number of samples.

[0115] The U-net network comprises downsampling and upsampling networks. The main task of the downsampling network is to progressively reduce the spatial resolution of the feature maps while increasing their depth to extract high-level abstract features from the input image. Each downsampling step typically includes two 3×3 convolutional layers, followed by a ReLU activation function. The convolutional layers extract image features, while the activation function introduces non-linear features. After every two convolutional layers, a 2×2 max-pooling operation is performed to reduce the spatial size of the feature maps. The max-pooling layer halves the width and height of the feature maps. The main task of the upsampling network is to progressively restore the spatial resolution of the feature maps by combining the high-level abstract features obtained in the downsampling stage with the local details of the original image to achieve accurate keypoint recognition. Each upsampling step includes an upsampling operation. First, a transposed convolution is performed to enlarge the spatial size of the feature maps, typically doubling their width and height. Then, skip connections are performed to concatenate the upsampled feature maps with those from the corresponding downsampling stage. After the skip connections, the feature maps pass through two 3×3 convolutional layers, with a ReLU activation function following each convolutional layer to further extract and fuse features.

[0116] The values ​​and functions of the parameters involved in the U-net network can be found in Table 4 below.

[0117] Table 4 S530 calculates the loss function based on the output keypoint location information and the labeled keypoint location information.

[0118] The loss function is used to measure the relative error between the output keypoint location information and the labeled keypoint location information.

[0119] In some embodiments, the loss function includes at least one of a first loss subfunction, a second loss subfunction, and a third loss subfunction.

[0120] The first loss function indicates the relative error between the vehicle keypoints in the second view image and the labeled vehicle keypoints in the output keypoint location information. The second loss function indicates the relative error between the keypoint locations in the first view image and the labeled vehicle keypoints in the output keypoint location information. The third loss function indicates the relative error between the x-coordinates of the vehicle keypoints in the first view image and the x-coordinates of the vehicle keypoints in the second view image in the output keypoint location information.

[0121] For example, the location of vehicle key points on the heatmap of the second view image. Marking points on the first view image Marking points on the second view image The key point is its position on the heat map of the first view image. .

[0122] At this point, the first loss function is represented by the following first calculation formula.

[0123] .

[0124] The second loss function is expressed by the following second calculation formula.

[0125] The third loss function is expressed by the following third calculation formula.

[0126] .

[0127] Optionally, the loss function is the sum of the first, second, and third loss sub-functions mentioned above. The loss function tong is expressed by the following fourth formula.

[0128] .

[0129] By training the neural network model using a hybrid loss function derived from the first, second, and third loss functions, the network optimization can be more comprehensively guided, improving the accuracy and consistency of keypoint detection.

[0130] S540 optimizes the parameters of the neural network model using an optimization algorithm to minimize the loss function.

[0131] Alternatively, the optimization algorithm can be a normalization algorithm, or it could be a backpropagation algorithm, etc.

[0132] S550 involves repeatedly calculating the loss function and optimizing the model parameters until the stopping iteration condition is met, at which point the iteration training stops and the key point recognition model is obtained.

[0133] The stopping condition for iteration can be that the loss function is less than a preset value, which is set based on experiments or experience; this embodiment of the application does not limit this. After the stopping condition, the number of iterations can be greater than a preset number, which is set based on experiments or experience.

[0134] By using the U-net network to train a neural network model, pixel-level key detection can be achieved, reducing the probability of key points being missed or falsely detected, thereby improving the accuracy of vehicle frame size calculation. In addition, the U-net network can better adapt to small sample tasks, and can train a more accurate key point recognition model even with a small number of samples.

[0135] This application also provides a vehicle frame structure design device 100, please refer to... Figure 7 It includes: an image acquisition module 710 for performing step S10; a key point recognition module 720 for performing step S20; a three-dimensional information determination module 730 for performing step S30; and a size determination module 740 for performing step S40.

[0136] In some embodiments, the device 100 further includes a model training module (not shown) for performing S50-S90.

[0137] This application also provides an electronic device 200, please refer to... Figure 8 It includes a processor 810 and a memory 820, wherein the memory 810 is used to store computer programs; the processor 820 is used to execute the programs stored in the memory 810 to implement the vehicle frame size determination method and / or key point recognition model training method described in any embodiment of this application.

[0138] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle frame size determination method and / or key point recognition model training method described in any embodiment of this application.

[0139] In this application, "multiple" refers to two or more.

[0140] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0141] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0142] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0143] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0144] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A vehicle frame structure design method, characterized in that, The method includes: Obtain the first view image and the second view image of the first model; By using a pre-trained keypoint recognition model to detect keypoints in the first view image and the second view image, the keypoint location information of the first view image and the keypoint location information of the second view image are obtained. Based on the key point location information of the first view image and the key point location information of the second view image, the three-dimensional information of multiple vehicle key points is determined. The dimensions of the vehicle frame corresponding to the first styling model are determined based on the three-dimensional information of the multiple vehicle key points.

2. The method according to claim 1, characterized in that, The determination of 3D information of multiple vehicle key points based on the key point location information of the first view image and the second view image includes: For each of the plurality of vehicle key points, the first coordinates of each vehicle key point in the first coordinate system are obtained based on the key point location information of the first view image. Based on the key point location information of the second view image, obtain the second coordinates of each vehicle key point in the second coordinate system; The three-dimensional information of each vehicle's key points is determined based on the first coordinate and the second coordinate.

3. The method according to claim 2, characterized in that, The first coordinate includes the first abscissa and the height coordinate of each vehicle key point in the first coordinate system, and the second coordinate includes the second abscissa and the ordinate of each vehicle key point in the second coordinate system. The determination of the three-dimensional information of each vehicle key point based on the first coordinate and the second coordinate includes: The average of the first and second abscissas is determined as the abscissa of each vehicle key point in the three-dimensional coordinate system, the ordinate is determined as the ordinate of each vehicle key point in the three-dimensional coordinate system, and the height coordinate is determined as the height coordinate of each vehicle key point in the three-dimensional coordinate system.

4. The method according to claim 1, characterized in that, The vehicle frame includes a first parameter, which is a length parameter; Determining the vehicle frame dimensions of the target vehicle model based on the three-dimensional coordinates of the multiple vehicle key points includes: The model length of the first parameter is determined based on the three-dimensional coordinates of at least two of the multiple vehicle key points. Obtain the scaling factor of the first parameter; The actual length of the first parameter is determined based on the scaling factor and the model length.

5. The method according to claim 4, characterized in that, The step of obtaining the scaling factor of the first parameter includes: Obtain the three-dimensional directions involved in the first parameter; When there is only one three-dimensional direction involved in the first parameter, the scaling factor corresponding to the three-dimensional direction involved in the third parameter is determined as the scaling factor of the first parameter. When the first parameter involves at least two three-dimensional directions, the scaling factor of the first parameter is calculated based on the scaling factors corresponding to the at least two three-dimensional directions involved in the first parameter.

6. The method according to claim 1, characterized in that, The vehicle frame includes a second parameter, which is an angle parameter; Determining the vehicle frame dimensions of the target vehicle model based on the three-dimensional coordinates of the multiple vehicle key points includes: Based on the three-dimensional coordinates of at least three vehicle key points among the plurality of vehicle key points, determine the vectors corresponding to the two sides of the second parameter respectively; The actual angle of the second parameter is determined based on the vectors corresponding to the two sides.

7. The method according to any one of claims 1 to 6, characterized in that, The training process of the key point recognition model includes: Multiple sets of training images are acquired, each set of training images including at least a first view image and a second view image of the second model, and the first view image and the second view image of the vehicle of the second model are respectively labeled with at least one vehicle key point; Each set of training images is input into the neural network model to obtain the key point location information of the output; The loss function is calculated based on the output vehicle key point information and the labeled vehicle key points. The parameters of the neural network model are optimized using an optimization algorithm to minimize the loss function; The steps of calculating the loss function and optimizing the model parameters are repeated until the stopping iteration condition is met, at which point the key point recognition model is obtained.

8. The method according to claim 7, characterized in that, The loss function includes at least one of a first loss sub-function, a second loss sub-function, and a third loss function; The first loss function is used to indicate the relative error between the vehicle key points in the second view image and the labeled vehicle key points in the output key point location information; The second loss function is used to indicate the relative error between the key point location information of the first view image in the output key point location information and the labeled vehicle key points; The third loss function is used to indicate the relative error between the abscissa of the vehicle key point in the first view image in the output key point location information and the abscissa of the vehicle key point in the second view image in the output key point location information.

9. The method according to any one of claims 1 to 6, characterized in that, The process of obtaining the first view image and the second view image of the first model includes: Obtain the original first view image and the original second view image of the first vehicle model; The original first view image is subjected to grayscale processing to obtain a first grayscale image; and the original second view image is subjected to grayscale processing to obtain a second grayscale image; the first grayscale image and the second grayscale image are the inputs of the key point recognition model.

10. The method according to claim 9, characterized in that, The original first view image is processed to obtain a first grayscale image; Furthermore, after performing grayscale processing on the original second view image to obtain a second grayscale image, the method further includes: The first grayscale image is downscaled to obtain the first view image; Furthermore, the second grayscale image is downscaled to obtain the second view image.

11. A vehicle frame structure design device, characterized in that, The device includes: The image acquisition module is used to acquire the first view image and the second view image of the first model; The key point recognition module is used to process the first view image through a pre-trained key point recognition model to obtain the key point location information of the first view image, and to process the second view image through the key point recognition model to obtain the key point location information of the second view image. The 3D information determination module is used to determine the 3D information of multiple vehicle key points based on the key point location information of the first view image and the key point location information of the second view image. The size determination module is used to determine the size of the vehicle frame corresponding to the first styling model based on the three-dimensional information of the multiple vehicle key points.

12. An electronic device, characterized in that, Including processor and memory, among which, The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory to implement the method described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-10.

Images