Photographing methods, photographing devices, equipment, storage media and software products

Abstract

The present disclosure relates to a photographing method, a photographing device, equipment, a storage medium and a program product. The method comprises: in the case that an electronic device enters a photographing preview stage, determining first key point information of a target vehicle and first attitude information of the target vehicle based on a preview image obtained for the target vehicle; wherein the key point information is used to indicate the spatial coordinates of each component of the vehicle; determining a target template image from a preset template image based on the first key point information and the first attitude information; wherein the preset template image comprises a template vehicle; displaying an alignment prompt on the preview image based on the position information of the template vehicle in the target template image, thereby reducing the problem of low matching accuracy of the target template image caused by the occlusion of part of the key points of the vehicle due to the photographing angle, and facilitating the improvement of the accuracy of determining the target template image, so as to ensure the accuracy of the user's composition and improve the user's photographing experience.

Description

Technical Field

[0001] This disclosure relates to the field of photography technology, and in particular to a photography method, photography device, equipment, storage medium, and program product. Background Technology

[0002] With the increasing popularity of automobiles and the rapid growth in user demand for vehicle photography, intelligent composition technology for vehicles has become a research hotspot. However, most users lack professional photography skills and cannot effectively utilize composition. Therefore, providing composition templates during vehicle photography can optimize the quality of user-generated images.

[0003] However, when the angle of the vehicle to be photographed or the shooting scene is complex and varied, the terminal cannot accurately provide the user with a composition template that matches the current scene, thus failing to guarantee the accuracy of the user's composition and failing to improve the user's photography experience. Summary of the Invention

[0004] To overcome the problems in related technologies, this disclosure provides a photographing method, photographing device, equipment, storage medium, and program product, which reduces the problem of low matching accuracy of target template images caused by the occlusion of key points of the vehicle due to the shooting angle. This helps to improve the accuracy of determining the target template image, thereby ensuring the accuracy of the user's composition and improving the user's photographing experience.

[0005] According to a first aspect of the present disclosure, a method for taking a picture is provided, comprising:

[0006] When the electronic device enters the shooting preview stage, based on the preview image acquired for the target vehicle, the first key point information and the first posture information of the target vehicle are determined; wherein, the key point information is used to indicate the spatial coordinates of various components of the vehicle.

[0007] Based on the first key point information and the first posture information, a target template image is determined from a preset template image; wherein, the preset template image includes a template vehicle;

[0008] Based on the position information of the template vehicle in the target template image, alignment prompts are displayed on the preview image; wherein, the alignment prompts are used to indicate the shooting angle of the electronic device.

[0009] In some embodiments, determining the target template image from the preset template image based on the first key point information and the first pose information includes:

[0010] Determine the second key point information and the second posture information of the template vehicle in each of the preset template images;

[0011] For each of the preset template images, a first matching degree between the target vehicle and the template vehicle is determined based on the first key point information, the first posture information, the second key point information, and the second posture information.

[0012] Based on each of the first matching degrees, the target template image is determined from the preset template images.

[0013] In some embodiments, determining the first matching degree between the target vehicle and the template vehicle based on the first key point information, the first posture information, the second key point information, and the second posture information includes:

[0014] Determine the first similarity between the first key point information and the second key point information;

[0015] Based on the angle information indicated by the first posture information, a first target point is determined from the first key point information, and based on the angle information indicated by the second posture information, a second target point is determined from the second key point information; wherein, the target point information is used to indicate the spatial coordinates of key components of the vehicle displayed in the image;

[0016] Determine the second similarity between the first target point information and the second target point information;

[0017] The first matching degree is determined based on the first similarity and the second similarity.

[0018] In some embodiments, determining the second similarity between the first target point information and the second target point information includes:

[0019] From each of the second target point information, select the third target point information that matches each of the first target point information;

[0020] A first vector information associated with the first target point information and a second vector information associated with the third target point information are determined; wherein the vector information is used to characterize the attitude information of the vehicle;

[0021] For each first target point information and the matched third target point information, a third similarity is determined based on the first vector information and the second vector information;

[0022] The second similarity is determined based on each of the third similarities.

[0023] In some embodiments, determining the third similarity based on the first vector information and the second vector information includes:

[0024] The cosine similarity between the first vector information and the second vector information is determined as the third similarity.

[0025] The determination of the second similarity based on each of the third similarities includes:

[0026] The second similarity is determined based on the average value corresponding to each of the cosine similarities.

[0027] In some embodiments, the method further includes:

[0028] Determine the first proportion of the target vehicle in the preview image, and the second proportion of each template vehicle in the corresponding preset template image;

[0029] For each of the preset template images, a second matching degree between the target vehicle and the template vehicle is determined based on the first proportion and the second proportion; wherein, the second matching degree characterizes the similarity between the shooting distance of the target vehicle and the shooting distance of the template vehicle;

[0030] Determining the target template image from the preset template images based on each of the first matching degrees includes:

[0031] The target template image is determined from the preset template image based on each of the first matching degree and each of the second matching degree.

[0032] In some embodiments, determining the target template image from the preset template image based on each of the first matching degrees and each of the second matching degrees includes:

[0033] For each of the preset template images, the first matching degree and the second matching degree are weighted based on preset weight values ​​to determine the target matching degree;

[0034] The preset template image corresponding to the highest target matching degree among all the target matching degrees is determined as the target template image.

[0035] In some embodiments, determining the first key point information and the first attitude information of the target vehicle based on a preview image acquired for the target vehicle includes:

[0036] Extract the image region containing the target vehicle from the preview image;

[0037] The size of the image region is adjusted to a preset size to obtain the image to be processed;

[0038] The image to be processed is input into the target recognition model to obtain the first key point information and the first pose information;

[0039] The target recognition model is used to identify key point information and attitude information of vehicles in the image.

[0040] In some embodiments, the method further includes:

[0041] Acquire sample images of a preset vehicle, third key point information of the preset vehicle, third attitude information of the preset vehicle, and a target heat map generated based on the third key point information.

[0042] The sample image is input into the initial recognition model to be trained, and the initial recognition model outputs predicted key point information, predicted pose information, and predicted heatmap.

[0043] Based on the differences between the predicted key point information and the third key point information, the differences between the predicted pose information and the third pose information, and the differences between the predicted heatmap and the target heatmap, the network parameters of the initial recognition model are updated.

[0044] The target recognition model is obtained when the updated network parameters meet the preset convergence conditions.

[0045] In some embodiments, inputting the sample image into an initial recognition model to be trained, wherein the initial recognition model outputs predicted keypoint information, predicted pose information, and predicted heatmap, includes:

[0046] The feature extraction network of the initial recognition model is used to extract the feature information of the preset vehicle;

[0047] The feature information is identified using the generative network of the initial recognition model to obtain the predicted heatmap; wherein, the generative network is used to determine the distribution information of key point information of the vehicle.

[0048] The feature information is identified using the recognition network of the initial recognition model to obtain the predicted key point information;

[0049] The feature information is identified using the classification network of the initial recognition model to obtain the predicted pose information.

[0050] According to a second aspect of the present disclosure, a photographing device is provided, comprising:

[0051] The information determination module is configured to determine the first key point information and the first posture information of the target vehicle based on the preview image acquired for the target vehicle when the electronic device enters the shooting preview stage; wherein, the key point information is used to indicate the spatial coordinates of various components of the vehicle.

[0052] The template determination module is configured to determine a target template image from a preset template image based on the first key point information and the first posture information; wherein the preset template image includes a template vehicle;

[0053] The prompting module is configured to display alignment prompts on the preview image based on the position information of the template vehicle in the target template image; wherein the alignment prompts are used to indicate the adjustment of the shooting angle of the electronic device.

[0054] In some embodiments, the template determining module includes:

[0055] The first submodule is configured to determine the second key point information of the template vehicle and the second posture information of the template vehicle in each of the preset template images;

[0056] The second submodule is configured to determine the first matching degree between the target vehicle and the template vehicle for each preset template image based on the first key point information, the first posture information, the second key point information, and the second posture information.

[0057] The third submodule is configured to determine the target template image from the preset template images based on each of the first matching degrees.

[0058] In some embodiments, the second submodule is specifically configured as follows:

[0059] Determine the first similarity between the first key point information and the second key point information;

[0060] Based on the angle information indicated by the first posture information, a first target point is determined from the first key point information, and based on the angle information indicated by the second posture information, a second target point is determined from the second key point information; wherein, the target point information is used to indicate the spatial coordinates of key components of the vehicle displayed in the image;

[0061] Determine the second similarity between the first target point information and the second target point information;

[0062] The first matching degree is determined based on the first similarity and the second similarity.

[0063] In some embodiments, the second submodule includes:

[0064] The selection module is configured to select third target point information that matches each of the first target point information from each of the second target point information.

[0065] The vector determination module is configured to determine first vector information associated with the first target point information and second vector information associated with the third target point information; wherein the vector information is used to characterize the attitude information of the vehicle.

[0066] The first similarity determination module is configured to determine a third similarity for each first target point information and the matched third target point information, based on the first vector information and the second vector information;

[0067] The second similarity determination module is configured to determine the second similarity based on each of the third similarities.

[0068] In some embodiments, the first similarity determination module is specifically configured as follows:

[0069] The cosine similarity between the first vector information and the second vector information is determined as the third similarity.

[0070] The determination of the second similarity based on each of the third similarities includes:

[0071] The second similarity is determined based on the average value corresponding to each of the cosine similarities.

[0072] In some embodiments, the apparatus further includes:

[0073] The proportion determination module is configured to determine the first proportion of the target vehicle in the preview image, and the second proportion of each template vehicle in the corresponding preset template image;

[0074] The processing module is configured to determine a second matching degree between the target vehicle and the template vehicle for each preset template image, based on the first proportion and the second proportion; wherein the second matching degree characterizes the similarity between the shooting distance of the target vehicle and the shooting distance of the template vehicle.

[0075] The third submodule is also configured as follows:

[0076] The target template image is determined from the preset template image based on each of the first matching degree and each of the second matching degree.

[0077] In some embodiments, the third submodule is further configured as follows:

[0078] For each of the preset template images, the first matching degree and the second matching degree are weighted based on preset weight values ​​to determine the target matching degree;

[0079] The preset template image corresponding to the highest target matching degree among all the target matching degrees is determined as the target template image.

[0080] In some embodiments, the information determining module is specifically configured as follows:

[0081] Extract the image region containing the target vehicle from the preview image;

[0082] The size of the image region is adjusted to a preset size to obtain the image to be processed;

[0083] The image to be processed is input into the target recognition model to obtain the first key point information and the first pose information;

[0084] The target recognition model is used to identify key point information and attitude information of vehicles in the image.

[0085] In some embodiments, the apparatus further includes:

[0086] The first acquisition module is configured to acquire sample images taken of a preset vehicle, third key point information of the preset vehicle, third posture information of the preset vehicle, and a target heat map generated based on the third key point information.

[0087] The second acquisition module is configured to input the sample image into the initial recognition model to be trained, and the initial recognition model outputs predicted key point information, predicted pose information and predicted heat map.

[0088] The update module is configured to update the network parameters of the initial recognition model based on the difference information between the predicted key point information and the third key point information, the difference information between the predicted pose information and the third pose information, and the difference information between the predicted heatmap and the target heatmap.

[0089] The generation module is configured to obtain the target recognition model when the updated network parameters meet the preset convergence conditions.

[0090] In some embodiments, the second acquisition module is specifically configured as follows:

[0091] The feature extraction network of the initial recognition model is used to extract the feature information of the preset vehicle;

[0092] The feature information is identified using the generative network of the initial recognition model to obtain the predicted heatmap; wherein, the generative network is used to determine the distribution information of key point information of the vehicle.

[0093] The feature information is identified using the recognition network of the initial recognition model to obtain the predicted key point information;

[0094] The feature information is identified using the classification network of the initial recognition model to obtain the predicted pose information.

[0095] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0096] processor;

[0097] Memory used to store computer programs or instructions;

[0098] The processor executes computer programs or instructions to implement the steps in any of the photographing methods described in the first aspect above.

[0099] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, comprising:

[0100] When a computer program or instruction in a storage medium is executed by a processor, the steps in any of the photographing methods described in the first aspect above are implemented.

[0101] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of any of the photographing methods in the first aspect described above.

[0102] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0103] In this embodiment of the present disclosure, when the electronic device enters the shooting preview stage, the first key point information and the first posture information of the target vehicle are first determined based on the preview image acquired for the target vehicle; and the target template image is determined from the preset template image based on the first key point information and the first posture information; and the alignment prompt is displayed on the preview image based on the position information of the template vehicle in the target template image.

[0104] On the one hand, based on the preview image, the first key point information and the first posture information of the target vehicle can be determined, which can reduce the problem of low matching accuracy of the target template image due to the occlusion of some key points of the vehicle caused by the shooting angle when matching the preset template image, and help improve the accuracy of determining the target template image.

[0105] On the other hand, when the target template image is determined, alignment prompts can be generated so that users can adjust the shooting angle of the electronic device based on the alignment prompts, thereby ensuring the accuracy of the user's composition and improving the user's photography experience.

[0106] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0107] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0108] Figure 1 This is a flowchart illustrating a photographing method according to an exemplary embodiment. Figure 1 .

[0109] Figure 2 This is a schematic diagram illustrating key point information of a vehicle according to an exemplary embodiment.

[0110] Figure 3 This is a schematic diagram illustrating the structure of an identification model according to an exemplary embodiment.

[0111] Figure 4 This is a flowchart illustrating a photographing method according to an exemplary embodiment. Figure 2 .

[0112] Figure 5 This is a block diagram illustrating a photographing device according to an exemplary embodiment.

[0113] Figure 6 This is a structural block diagram of an electronic device 600 according to an exemplary embodiment. Detailed Implementation

[0114] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0115] Figure 1 This is a flowchart illustrating a photographing method according to an exemplary embodiment. Figure 1 ,like Figure 1 As shown, this photo-taking method mainly includes the following steps:

[0116] In step 101, when the electronic device enters the shooting preview stage, the first key point information and the first posture information of the target vehicle are determined based on the preview image acquired for the target vehicle; wherein, the key point information is used to indicate the spatial coordinates of each component of the vehicle.

[0117] In step 102, a target template image is determined from a preset template image based on the first key point information and the first posture information; wherein, the preset template image includes a template vehicle;

[0118] In step 103, based on the position information of the template vehicle in the target template image, alignment prompts are displayed on the preview image; wherein, the alignment prompts are used to indicate the shooting angle of the electronic device.

[0119] It should be noted that the photographing method proposed in this disclosure can be applied to electronic devices. Here, electronic devices can include terminal devices, such as mobile terminals or fixed terminals. Mobile terminals can include mobile phones, tablets, laptops, wearable electronic devices, etc. Fixed terminals can include desktop computers, smart TVs, in-vehicle systems, etc. In some other embodiments, the photographing method can also be applied to applications installed on electronic devices.

[0120] In other embodiments, the photographing method described in this disclosure can be configured in a photographing device, which can be located in an electronic device; this disclosure does not limit this. It should be noted that the execution entity of this disclosure can be a central processing unit (CPU) in the electronic device in hardware, and related background services in the electronic device in software; this is not limited.

[0121] In some embodiments, most users lack professional photography skills. Providing guidance during shooting, such as intelligent vehicle template matching, can optimize the quality of the user's photos and subtly cultivate their compositional awareness. For example, during the shooting process, the template vehicle in the target template image is positioned within the rule of thirds, allowing the user to imitate it and obtain a target photo with the vehicle positioned within the rule of thirds.

[0122] In some embodiments, the intelligent vehicle template matching method includes two main steps: evaluating the pose information of the vehicle to be photographed; and determining a target template image that matches the current shooting scene. However, in implementation, the vehicle's angle information is complex and variable. For example, the vehicle may appear in the image from different perspectives, such as a frontal view, a side view, a rear view, or a high-angle overhead view. These perspectives reduce the processing accuracy of the vehicle pose estimation algorithm and the accuracy of determining the target template image. Furthermore, due to the complexity of the shooting environment, the vehicle to be photographed may be obscured by other objects (e.g., pedestrians, trees, or other vehicles), which also affects the accuracy of vehicle pose estimation.

[0123] In this embodiment of the present disclosure, during the preview stage of the electronic device's shooting, a preview image of the target vehicle is acquired; and based on the preview image, first key point information and first posture information of the target vehicle are determined; then, based on the first key point information and the first posture information, a target template image is determined from a preset template image. This not only reduces the problem of low matching accuracy of the target template image due to partial occlusion of key points of the vehicle caused by the shooting angle, but also improves the accuracy of vehicle posture estimation.

[0124] Here, the target vehicle is the vehicle to be photographed in the current scene.

[0125] In some embodiments, during the preview phase of the electronic device, the electronic device can acquire at least one preview image of the target vehicle. To improve the accuracy of determining the first key point information of the target vehicle, when the preview image is a single frame, it can be preprocessed, for example, by grayscale conversion or image enhancement, and the first key point information can be determined based on the preprocessed preview image. When the preview image consists of multiple frames, the index parameters of each preview image are first determined; then, preview images whose index parameters are all greater than a preset index threshold are filtered out; and finally, the first key point information is determined based on the filtered preview images; wherein, the index parameters are positively correlated with the image performance.

[0126] Here, the first key point information indicates the spatial coordinates of the various components of the vehicle in two-dimensional space. These components include those shown in the preview image and those not shown.

[0127] For example, the first key point information includes, but is not limited to, the coordinates of the wheel center, the coordinates of the headlights, the coordinates of the four contour points that make up the roof, and the coordinates of specific contour points on the body contour.

[0128] Here, the first attitude information is used to describe the placement position and orientation of the target vehicle in space. For example, when the vehicle is determined to be located at coordinates (x, y, z) and oriented at degrees θ, (x, y, z) represents the placement position of the vehicle, indicating its specific location in space; while θ represents the orientation of the vehicle, indicating the rotation angle of the vehicle relative to a certain reference direction (e.g., due north).

[0129] In some embodiments, a filter (such as a Gaussian filter) is used to smooth the preview image to reduce the impact of noise on edge detection; edge detection is performed on the preprocessed preview image using an edge detection algorithm; and then, based on shape matching and contour tracking algorithms, the first key point information of the target vehicle is identified from the edge-detected image.

[0130] In some embodiments, a preview image is input into a trained model to obtain the first key point information of the target vehicle; a 3D bounding box or skeleton model of the vehicle is constructed based on the first key point information; and the first pose information of the vehicle is determined based on the geometric constraints in the 3D bounding box or skeleton model, such as the relative positions, angles, and distances between the first key points.

[0131] In other embodiments, in order to improve the efficiency of determining the first posture information, a recognition model capable of recognizing the posture information of a vehicle is trained in advance based on sample images of a preset vehicle and the posture information of the preset vehicle; the preview image is input into the recognition model, the recognition model extracts the image features of the target vehicle, recognizes the image features, and outputs the first posture information.

[0132] In this embodiment of the present disclosure, the preset template image includes a template vehicle. When the first posture information and the first key point information are determined, the first posture information of the target vehicle and the posture information of the template vehicle are matched, and the first key point information of the target vehicle and the key point information of the template vehicle are matched, so as to determine the target template image from the preset template image, thereby improving the accuracy of determining the target template image.

[0133] Here, preset template images are used to provide users with preset shooting angles and layouts, which can help users quickly determine the shooting position and direction of the target vehicle, saving a lot of trial and error time. The number of preset template images can be arbitrarily set according to needs, for example, 10; this embodiment does not limit this. Furthermore, the preset template images can be provided by the electronic device manufacturer or downloaded by the user through a third-party application; this embodiment also does not limit this.

[0134] Understandably, when the target template image is determined, alignment prompts are displayed on the preview image based on the position information of the template vehicle in the target template image, in order to guide users to take photos that are more in line with the aesthetic composition.

[0135] Here, the display position, size, and format of the alignment prompt can be set arbitrarily according to requirements. For example, the target template image can be displayed in the upper left corner of the preview image; or a dashed frame can be formed in the preview image, and the position and size of the virtual frame in the preview image can match the position and size of the template vehicle in the target template image.

[0136] In this embodiment of the present disclosure, when the electronic device enters the shooting preview stage, the first key point information and the first posture information of the target vehicle are first determined based on the preview image acquired for the target vehicle; and the target template image is determined from the preset template image based on the first key point information and the first posture information; and the alignment prompt is displayed on the preview image based on the position information of the template vehicle in the target template image.

[0137] On the one hand, based on the preview image, the first key point information and the first posture information of the target vehicle can be determined, which can reduce the problem of low matching accuracy of the target template image due to the occlusion of some key points of the vehicle caused by the shooting angle when matching the preset template image, and help improve the accuracy of determining the target template image.

[0138] On the other hand, when the target template image is determined, alignment prompts can be generated so that users can adjust the shooting angle of the electronic device based on the alignment prompts, thereby ensuring the accuracy of the user's composition and improving the user's photography experience.

[0139] In some embodiments, determining a target template image from a preset template image based on first key point information and first pose information includes:

[0140] Determine the second key point information and the second posture information of the template vehicle in each preset template image;

[0141] For each preset template image, the first matching degree between the target vehicle and the template vehicle is determined based on the first key point information, the first posture information, the second key point information, and the second posture information.

[0142] Based on each first matching degree, the target template image is determined from the preset template images.

[0143] It should be noted that, in order to improve the accuracy of determining the target template image, the second key information and the second pose information of the template vehicle in each preset template image can be determined first, so that the target vehicle and the template vehicle can be matched in the same dimension; then, based on the first key point information, the first pose information, the second key point information, and the second pose information, the first match degree between each template vehicle and the target vehicle can be determined.

[0144] In some embodiments, when determining the second key information and the second posture information of the template vehicle, a first similarity is first determined between the angle information indicated by the second posture information and the angle information indicated by the first posture information; then, a second similarity is determined between the second key information of the template vehicle and the first key information of the target vehicle; and then, based on the first similarity and the second similarity, a first matching degree between the target vehicle and the template vehicle is determined.

[0145] In other embodiments, when determining the second key information and the second posture information of the template vehicle, first target point information is determined from the first key point information based on the first posture information; and second target point information is determined from the second key information based on the second posture information; then, the similarity between the first target point information and the second target point information is determined to determine the first matching degree between the target vehicle and the template vehicle. Here, the target point information is used to indicate the spatial coordinates of the various components of the vehicle displayed in the image.

[0146] Here, when determining the first matching degree between the target vehicle and each template vehicle, the target template image can be determined based on each first matching degree.

[0147] In some embodiments, once each first matching degree is determined, the first matching degrees are sorted; based on the sorting results, the preset template image corresponding to the largest first matching degree among the first matching degrees is determined as the target template image.

[0148] In other embodiments, after determining each first matching degree, the first matching degrees are sorted; based on the sorting results, if it is determined that the differences between the larger first matching degrees are small, the preset template images corresponding to these first matching degrees are determined as candidate template images; and the first proportion of the target vehicle in the preview image and the second proportion of each template vehicle in the corresponding candidate template image are determined; for each candidate template image, the second matching degree between the target vehicle and the template vehicle is determined based on the first proportion and the second proportion; and the candidate template image corresponding to the largest second matching degree is determined as the target template image.

[0149] In this embodiment, the second key point information and the second posture information of the template vehicle are first determined; then, based on the first key point information, the first posture information, the second key point information, and the second posture information, the first matching degree between the target vehicle and the template vehicle is determined; then, based on each first matching degree, the target template image is determined from the preset template image, which improves the matching accuracy of the target template image and thus ensures the accuracy of the user's composition.

[0150] In some embodiments, determining a first matching degree between the target vehicle and the template vehicle based on first key point information, first posture information, second key point information, and second posture information includes:

[0151] Determine the first similarity between the first key point information and the second key point information;

[0152] Based on the angle information indicated by the first posture information, the first target point information is determined from the first key point information, and based on the angle information indicated by the second posture information, the second target point information is determined from the second key point information; wherein, the target point information is used to indicate the spatial coordinates of key components of the vehicle displayed in the image;

[0153] Determine the second similarity between the first target point information and the second target point information;

[0154] The first matching degree is determined based on the first similarity and the second similarity.

[0155] It should be explained that, considering that deviations in non-critical points of the target vehicle can also lead to low matching accuracy of the target template image, when determining the first key point information, the first target point information is determined from the first key point information based on the angle information indicated by the first posture information, so as to filter out non-critical points.

[0156] In some embodiments, if the angle information indicated by the first posture information belongs to a preset angle category, the first target point information is determined from the first key point information based on the preset angle category to which the angle information belongs.

[0157] In this embodiment of the disclosure, based on the angle information indicated by the second posture information, the second target point information is determined from the second key point information, so that the target vehicle and the template vehicle are determined to have a first matching degree in the same dimension, thereby improving the reliability of determining the first matching degree.

[0158] Here, target point information is used to indicate the spatial coordinates of key components of the vehicle displayed in the image. These key components include, but are not limited to, the vehicle's wheels, headlights, and roof.

[0159] For example, if the key components of the target vehicle displayed in the preview image are the left and right front wheels, the left front headlight, the right front headlight, the left rear headlight, and the left-front-right-rear roof, then the first target point information is the spatial coordinates of the aforementioned key components.

[0160] Understandably, in order to improve the accuracy of determining the first matching degree, the first similarity between the first key point information and the second key point information, and the second similarity between the first target point information and the second target point information can be determined; then, based on the first similarity and the second similarity, the first matching degree can be determined.

[0161] In some embodiments, when a first similarity and a second similarity are determined, the average value between the first similarity and the second similarity is calculated; the average value is then determined as the first matching degree.

[0162] In other embodiments, when determining the first similarity and the second similarity, the first similarity is weighted based on a first weight value to obtain a first weighted value; and the second similarity is weighted based on a second weight value to obtain a second weighted value; then, the first matching degree is obtained based on the first weighted value and the second weighted value. Here, the first weight value and the second weight value can be the same or different.

[0163] In this embodiment, target point information is determined from key point information based on angle information indicated by posture information; a first similarity between the first key point information and the second key point information, and a second similarity between the first target point information and the second target point information are determined; then, a first matching degree is determined based on the first similarity and the second similarity. This not only reduces the problem of low template matching accuracy caused by partial occlusion of key points of the vehicle due to the shooting angle, but also reduces the impact of low template matching accuracy caused by deviations of non-key points of the vehicle, thereby effectively improving the accuracy of the target template image.

[0164] In some embodiments, determining a second similarity between the first target point information and the second target point information includes:

[0165] From each of the second target point information, select the third target point information that matches each of the first target point information;

[0166] Determine first vector information associated with first target point information and second vector information associated with third target point information; wherein, the vector information is used to characterize the vehicle's attitude information;

[0167] For each first target point information and the matched third target point information, a third similarity is determined based on the first vector information and the second vector information;

[0168] The second similarity is determined based on each third similarity.

[0169] In some embodiments, during vehicle manufacturing, the various components of a vehicle (such as the front wheels, headlights, etc.) are installed according to fixed geometric relationships; therefore, the relative positions of the various components are correlated. Vectors include magnitude and direction. By determining vector information, the relative positional relationships between the various components of the vehicle and the vehicle's attitude information can be determined. For example, the vector from the left front wheel to the right front wheel reflects the relative position of the two front wheels, and the vector from the left front wheel to the left headlight reflects the relative position of the left front wheel relative to the left headlight. When the vehicle's attitude information changes, the direction of the vector changes; by analyzing this change in vector direction, the degree of change in the vehicle's attitude information can be inferred.

[0170] Therefore, in this embodiment of the present disclosure, in order to improve the accuracy of determining the second similarity, third target point information that matches each first target point information can be selected from each second target point information; for each first target point information and the matching third target point information, a third similarity is determined based on the first vector information and the second vector information; and then a second similarity is determined based on each third similarity.

[0171] Here, determining the third target point information that matches the first target point information can be understood as selecting, from the various key components displayed by the template vehicle in the preset template image, the component of the same type as the key component displayed by the target vehicle in the preview image.

[0172] For example, if the first target point information is the spatial coordinates of the left front wheel of the target vehicle, the spatial coordinates of the right front wheel of the target vehicle, and the spatial coordinates of the left front light of the target vehicle, then the third target point information that matches the first target point information is the spatial coordinates of the left front wheel of the template vehicle, the spatial coordinates of the right front wheel of the template vehicle, and the spatial coordinates of the left front light of the template vehicle.

[0173] Here, the first vector information associated with a target point can be one or more, and this embodiment of the disclosure does not limit this.

[0174] For example, if the first target point information is the left front headlight, then the first vector information associated with the left front headlight may include: left front headlight - right front headlight, left front headlight - left rear headlight, and left front headlight - left front roof.

[0175] In some embodiments, in order to improve the efficiency of determining the first vector information, preset vector information constructed from key point information is first determined; after determining the target point information from the key point information, vector information associated with the target point can be determined from the preset vector information.

[0176] For example, Figure 2This is a schematic diagram illustrating key point information of a vehicle according to an exemplary embodiment, such as... Figure 2 As shown in Figure 20, there are 12 key points of the vehicle. Figure 21 shows 20 sets of vectors that can represent the vehicle's attitude information. These 20 sets of vectors are specifically divided into: 6 sets of horizontal vectors (left and right front wheels, left and right rear wheels, left and right front lights, left and right rear lights, left and right front lights, left and right rear roofs), 6 sets of vertical vectors (left and right front and rear wheels, right and right front and rear wheels, left and right front and rear lights, right and right front and rear lights), and 8 sets of diagonal vectors (left front light - left front wheel, left front light - left front roof, right front light - right front wheel, right front light - right front roof, left rear light - left rear wheel, left rear light - left rear roof, right rear light - right rear wheel, right rear light - right rear roof).

[0177] Here, after determining the first vector information and the second vector information, a third similarity can be obtained based on the first vector information and the second vector information. There are many methods for determining the third similarity, such as the Euclidean distance similarity calculation method or the cosine similarity calculation method, and this embodiment of the disclosure does not limit the method.

[0178] In some embodiments, the Euclidean distance between the first vector information and the second vector information is first determined; then, based on the formula for converting Euclidean distance to similarity, the Euclidean distance is converted into a similarity; and then the converted similarity is determined as the third similarity.

[0179] For example, the first target point information is the coordinates of the left front light of the target vehicle, and the second target point information is the coordinates of the left front light of the template vehicle; based on the coordinates of the left front light of the target vehicle to the coordinates of the right front light of the target vehicle, the first vector information is determined; and based on the coordinates of the left front light of the template vehicle to the coordinates of the right front light of the template vehicle, the second vector information is determined; then the Euclidean distance between the first vector information and the second vector information is determined; and then the Euclidean distance is converted to a similarity to obtain a third similarity.

[0180] Here, when obtaining the third similarity between each first target point information and the matched third target point information, the second similarity can be determined based on each third similarity.

[0181] In some embodiments, once each third similarity is determined, the third similarities are sorted to obtain a sorting result; and the largest third similarity indicated by the sorting result is determined as the second similarity.

[0182] In other embodiments, after determining each third similarity, the average value corresponding to each third similarity is determined; and the determined average value is determined as the second similarity.

[0183] In some embodiments, to improve the accuracy of the first similarity, when determining the first keypoint information and the second keypoint information, a third vector information associated with the first keypoint information and a fourth vector information associated with the second keypoint information are determined; for each first keypoint information and matching second keypoint information, a fourth similarity is determined based on the third and fourth vector information; and a first similarity is determined based on each fourth similarity. Here, the method for determining the fourth similarity includes, but is not limited to, the Euclidean distance similarity calculation method or the cosine similarity calculation method.

[0184] In this embodiment of the present disclosure, third target point information that matches each first target point information is selected from each second target point information; for each first target point information and the matching third target point information, a third similarity is determined based on the first vector information and the second vector information; and a second similarity is determined based on each third similarity, thereby improving the accuracy of the second similarity and thus improving the accuracy of target template image matching.

[0185] In some embodiments, determining a third similarity based on the first vector information and the second vector information includes:

[0186] The cosine similarity between the first and second vector information is determined as the third similarity.

[0187] Based on each third similarity score, the second similarity score is determined, including:

[0188] The second similarity is determined based on the average value of each cosine similarity.

[0189] In some embodiments, the cosine similarity between two sets of vectors can be measured by calculating the cosine of the angle between the two vectors. The closer the cosine value is to 1, the closer the angle between the two vectors is to 0 degrees, and the more similar the two vectors are; the closer the cosine value is to 0, the closer the angle between the two vectors is to 90 degrees, and the less similar the two vectors are.

[0190] In this embodiment, the cosine similarity between the first and second vector information is determined as the third similarity; the average value corresponding to each cosine similarity is determined; and the average value is then determined as the second similarity. Since cosine similarity measures the angle between vectors and focuses on differences in direction, and the degree of difference in direction matches the degree of difference in the vehicle's attitude information, the similarity between the target vehicle and the template vehicle can be accurately assessed.

[0191] In some embodiments, when the first vector information and the second vector information are determined, the dot product of the first vector information and the second vector information is determined; and the magnitude of the first vector information and the magnitude of the second vector information are calculated respectively; then, based on the dot product and the two magnitudes, the cosine similarity is determined.

[0192] In this embodiment of the disclosure, the second similarity is obtained by calculating the cosine similarity between each first vector information and the matched second vector information, thereby improving the accuracy and efficiency of determining the second similarity.

[0193] In some embodiments, the method further includes:

[0194] Determine the first proportion of the target vehicle in the preview image, and the second proportion of each template vehicle in the corresponding preset template image;

[0195] For each preset template image, a second matching degree between the target vehicle and the template vehicle is determined based on a first ratio and a second ratio; wherein, the second matching degree characterizes the similarity between the shooting distance of the target vehicle and the shooting distance of the template vehicle.

[0196] Based on each first matching degree, the target template image is determined from the preset template image, including:

[0197] Based on each first matching degree and each second matching degree, the target template image is determined from the preset template images.

[0198] Understandably, the area ratio of a vehicle in an image can represent the vehicle's shooting distance. In addition to determining the first matching degree based on pose information and key point information, a second matching degree can be determined based on the first proportion of the target vehicle in the preview image and the second proportion of the template vehicle in the template image; then, based on the first and second matching degrees, the target template image is determined, thereby improving the accuracy of determining the target template image.

[0199] Here, the larger the proportion of the vehicle in the image, the closer the vehicle was photographed; the smaller the proportion of the vehicle in the image, the farther the vehicle was photographed.

[0200] In some embodiments, when a preview image is obtained, a first image region containing the target vehicle is extracted from the preview image; and a first proportion is obtained based on the area of ​​the first image region in the preview image. Similarly, when a preset template image is obtained, a second image region containing the template vehicle is extracted from the preset template image; and a second proportion is obtained based on the area of ​​the second image region in the preview image.

[0201] In this embodiment of the disclosure, for each preset template image, a second matching degree, representing the similarity between the shooting distance of the target vehicle and the shooting distance of the template vehicle, is first determined; then, based on each first matching degree and each second matching degree, a target template image is determined from the preset template images. Thus, by comparing the area ratio of vehicles in the image, the similarity of the vehicle shooting distances is determined, further filtering the preset template images, improving the accuracy of template matching, and thereby improving the accuracy of the target template image.

[0202] In some embodiments, determining a target template image from a preset template image based on each first matching degree and each second matching degree includes:

[0203] For each preset template image, the first matching degree and the second matching degree are weighted based on preset weight values ​​to determine the target matching degree.

[0204] The preset template image corresponding to the target with the highest target matching degree is determined as the target template image.

[0205] In this embodiment of the disclosure, a weight value is preset. When the first matching degree and the second matching degree are determined, the first matching degree and the second matching degree are weighted based on the preset weight value, so that the first matching degree and the second matching degree are scaled to the same scale after weighting, thereby reducing the problem that the target matching degree is inaccurate due to the large difference between the first matching degree and the second matching degree.

[0206] Here, the preset weight value for the first matching degree and the preset weight value for the second matching degree can be the same, for example, both are 0.5; or they can be different, for example, the preset weight value for the first matching degree is 0.8 and the preset weight value for the second matching degree is 0.2. This disclosure does not limit this.

[0207] It is understandable that, given the target matching degree corresponding to each preset template image, the target matching degrees can be sorted to obtain a sorting result; and the preset template image corresponding to the maximum target matching degree indicated by the sorting result can be determined as the target template image, thereby improving the matching accuracy of the target template image.

[0208] For example, the formula for calculating the target matching degree between the target vehicle and the template vehicle can be as follows:

[0209] similarity=0.8*(0.5*similarity1+0.5*similarity2)+0.2*similarity3 (1);

[0210] In formula (1), similarity is the target matching degree, similarity1 is the first similarity, similarity2 is the second similarity, similarity3 is the second matching degree, 0.8 is the preset weight value for the first matching degree, and 0.2 is the preset weight value for the second matching degree.

[0211] In this embodiment of the present disclosure, the target matching degree corresponding to each preset template image is first determined; then the preset template image corresponding to the maximum target matching degree among all target matching degrees is determined as the target template image, thereby improving the accuracy and efficiency of the target template image.

[0212] In some embodiments, based on a preview image acquired for the target vehicle, determining first key point information and first attitude information of the target vehicle includes:

[0213] Extract the image region containing the target vehicle from the preview image;

[0214] Adjust the size of the image area to the preset size to obtain the image to be processed;

[0215] The image to be processed is input into the target recognition model to obtain the first key point information and the first pose information;

[0216] Among them, the target recognition model is used to identify key point information and attitude information of vehicles in the image.

[0217] It should be noted that, in order to improve the accuracy of determining the first key point information and the first pose information, the preview image can be input into the target recognition model, which will then identify the image features of the preview image and output the first key point information and the first pose information.

[0218] In this embodiment of the disclosure, considering that there is noise interference in the preview image, which is not conducive to improving the accuracy of the target recognition model, when the preview image is obtained, the image region containing the target vehicle is first extracted from the preview image to remove the background interference and highlight the image features of the target vehicle; and the size of the image region is adjusted to a preset size to obtain the image to be processed, so as to improve the generalization ability of the target recognition model.

[0219] Here, the preset size can be set arbitrarily according to requirements, and this embodiment does not limit it.

[0220] For example, an image size of 224×224 is beneficial for the object recognition model to better learn and recognize features in the image. At the same time, 224×224 is a relatively small image size, which can reduce the computational load and memory usage of the object recognition model, thereby improving processing speed and efficiency. Therefore, the preset size can be set to 224×224.

[0221] In some embodiments, in order to improve the accuracy of determining the second key point information and the second posture information, when a preset template image is obtained, a first image region containing the template vehicle is extracted from the preset template image; the size of the first image region is adjusted to a preset size to obtain a first image to be processed; then the first image to be processed is input into the target recognition model, and the target recognition model outputs the second key point information and the second posture information.

[0222] In this embodiment of the disclosure, by preprocessing the preview image to obtain the image to be processed, the interference of noise can be reduced, thereby improving the generalization ability of the target recognition model; then the image to be processed is input into the target recognition model to obtain the first key point information and the first pose information, thereby improving the accuracy and efficiency of determining the first key point information and the first pose information.

[0223] In some embodiments, the method further includes:

[0224] Acquire sample images of a preset vehicle, third key point information of the preset vehicle, third attitude information of the preset vehicle, and target heat map generated based on the third key point information.

[0225] The sample image is input into the initial recognition model to be trained. The initial recognition model outputs predicted key point information, predicted pose information, and predicted heatmap.

[0226] Based on the differences between predicted key point information and third key point information, the differences between predicted pose information and third pose information, and the differences between predicted heatmap and target heatmap, the network parameters of the initial recognition model are updated.

[0227] Once the updated network parameters meet the preset convergence conditions, the target recognition model is obtained.

[0228] It should be noted that, in order to improve the recognition accuracy of the target recognition model, in the initial recognition model training process, in addition to acquiring sample images of the preset vehicle, the third key point information of the preset vehicle, and the third pose information of the preset vehicle, a target heat map generated based on the third key point information can also be acquired. This allows the model to capture the blur and uncertainty of local areas in the image, thereby improving the initial recognition model's ability to recognize the key point information and pose information of the vehicle.

[0229] In this embodiment of the present disclosure, when a sample image is input into the initial recognition model, the initial recognition model can extract the image features of a preset vehicle in the sample image; and recognize the image features and output the recognition results, namely, predicted key point information, predicted posture information and predicted heat map.

[0230] Understandably, different loss functions exhibit varying sensitivities to the differences between labeled data (i.e., the aforementioned third keypoint information, third pose information, and target heatmap) and predicted data (i.e., predicted keypoint information, predicted pose information, and predicted heatmap). To improve the accuracy of the initial recognition model training, different loss functions can be used to measure the differences between predicted keypoint information and third keypoint information, between predicted pose information and third pose information, and between predicted heatmap and target heatmap, respectively.

[0231] Here, based on the aforementioned difference information, the network parameters of the initial recognition model are adjusted; and if the updated network parameters meet the preset convergence conditions, it is determined that the initial recognition model has completed training and can be identified as the target recognition model. The network parameters include, but are not limited to, the model's weights and biases. The preset convergence conditions include, but are not limited to, loss convergence, reaching the maximum number of iterations, or the norm of the gradient being less than a preset threshold.

[0232] In some embodiments, each third keypoint in the labeled data is rendered as a Gaussian plot, where the mean of the Gaussian plot corresponds to the location of the third keypoint, and the variance determines the ambiguity (i.e., uncertainty) of the heatmap. The initial recognition model outputs K feature maps, each corresponding to a predicted keypoint. These feature maps, after appropriate post-processing (e.g., argmax operation), can yield the specific locations of the preset keypoints. During training, the initial recognition model learns the location distribution of keypoints by minimizing the difference between the predicted heatmap and the target heatmap (e.g., mean squared error loss).

[0233] For example, the formula for calculating the first loss function, which measures the difference between the predicted heatmap and the target heatmap, can be as follows:

[0234]

[0235] In formula (2), MSE represents the difference information between the predicted heatmap and the target heatmap. To predict the heat map, H i This is the target heatmap. Here, the smaller the MSE value, the higher the prediction accuracy of the initial identification model.

[0236] The formula for calculating the second loss function, which measures the difference between predicted keypoint information and third keypoint information, can be as follows:

[0237]

[0238] In formula (3), x represents the difference between the predicted key point information and the third key point information; w is a parameter that controls the shape of the loss function; ε is a smoothing parameter used to control the smoothness of the curve in the small error region; C is a constant used to make the Wing loss function continuous at |x|=w.

[0239] Here, Wing loss can improve the imbalance in how some loss functions (such as MSE or MAE) handle large and small errors. By segmenting the error, Wing loss can use a similar metric to MAE in the large error region, thereby reducing the impact of large errors on the overall performance evaluation and thus balancing the accuracy and robustness of key point information during detection.

[0240] The formula for calculating the third loss function, which measures the difference between the predicted pose information and the third pose information, can be as follows:

[0241]

[0242] In formula (4), N is the number of categories, and y i This represents the probability distribution of the third pose information. To predict the probability distribution of attitude information.

[0243] In this embodiment, sample images, third key point information, third pose information, and target heatmaps are first acquired; then, the sample images are input into an initial recognition model to obtain predicted key point information, predicted pose information, and predicted heatmaps; by determining the differences, the network parameters of the initial recognition model are updated to obtain a target recognition model, thereby improving the accuracy of training the target recognition model and thus improving the precision of the target recognition model.

[0244] In some embodiments, a sample image is input into an initial recognition model to be trained, and the initial recognition model outputs predicted keypoint information, predicted pose information, and a predicted heatmap, including:

[0245] The feature extraction network of the initial recognition model is used to extract the feature information of the preset vehicle;

[0246] The generative network of the initial recognition model is used to identify feature information and obtain a predicted heatmap; the generative network is used to determine the distribution information of key points of the vehicle.

[0247] The feature information is identified using the recognition network of the initial recognition model to obtain the predicted key point information;

[0248] The classification network of the initial recognition model is used to identify feature information and obtain predicted pose information.

[0249] Here, the feature extraction network can be a Convolutional Neural Network (CNN) or a Transformer model, etc.; the feature extraction network includes at least convolutional layers and pooling layers. The number of convolutional and pooling layers within the feature extraction network can be arbitrarily set according to requirements, and this embodiment does not limit this.

[0250] In some embodiments, during the initial training of the recognition model, the network parameters of the generator network, the recognition network, and the classification network are adjusted, while the network parameters of the feature extraction network remain unchanged, so as to achieve the effect of training only the generator network, the recognition network, and the classification network, which is beneficial to improving the generalization ability of the classification model.

[0251] For example, Figure 3 This is a schematic diagram illustrating the structure of a recognition model according to an exemplary embodiment, such as... Figure 3 As shown, the basic backbone network of the initial recognition model is a MobileNetv2 structure. To improve representation capabilities, the feature extraction network 31 of the initial recognition model includes an ASPP (Atrous Spatial Pyramid Pooling) module in addition to convolutional layers. The input sample image 30 is processed by convolutional layers to extract features and obtain the initial feature information of the preset vehicle. The APSS module performs multi-scale extraction on the initial feature information and performs feature fusion to obtain the target features. The generator network 32 (Lmks Hp head) of the initial recognition model identifies the target features and obtains the predicted heatmap 35. The recognition network 33 (Lmks Reg head) of the initial recognition model identifies the target features and obtains the predicted key point information 36. The classification network 34 (Angle cls head) of the initial recognition model identifies the target features and obtains the predicted pose information 37.

[0252] Here, the Lmks Hp head fully utilizes the heatmap method to capture the advantages of ambiguity and uncertainty in local areas of the image, thereby improving the initial recognition model's ability to evaluate the vehicle's posture information; the LmksReg head can regress and output key point information representing the vehicle, thereby improving the accuracy and efficiency of determining key point information; the Angle cls head can output the vehicle's posture information, thereby improving the accuracy and efficiency of determining the target template image.

[0253] In this embodiment of the disclosure, by using different networks of the initial recognition model, predicted key point information, predicted pose information, and predicted heatmap are obtained, which helps to improve the accuracy of different predicted information, thereby improving the training accuracy of the initial recognition model.

[0254] Figure 4 This is a flowchart illustrating a photographing method according to an exemplary embodiment. Figure 2 ,like Figure 4 As shown, this photo-taking method includes at least the following steps:

[0255] In step 401, when the electronic device enters the shooting preview stage, the preview image acquired for the target vehicle is input into the target recognition model to obtain the first key point information and the first posture information.

[0256] In step 402, a first similarity is determined between the first key point information and the second key point information.

[0257] In step 403, based on the angle information indicated by the first attitude information, the first target point information is determined from the first key point information, and based on the angle information indicated by the second attitude information, the second target point information is determined from the second key point information.

[0258] In step 404, a second similarity is determined between the first target point information and the second target point information.

[0259] In step 405, a first matching degree is determined based on the first similarity and the second similarity.

[0260] In step 406, the first proportion of the target vehicle in the preview image and the second proportion of each template vehicle in the corresponding preset template image are determined.

[0261] In step 407, a second matching degree is determined based on the first proportion and the second proportion.

[0262] In step 408, the first matching degree and the second matching degree are weighted based on preset weight values ​​to determine the target matching degree.

[0263] In step 409, the preset template image corresponding to the target matching degree with the highest target matching degree is determined as the target template image.

[0264] In step 410, based on the position information of the template vehicle in the target template image, alignment prompts are displayed on the preview image; wherein, the alignment prompts are used to indicate the shooting angle of the electronic device.

[0265] On the one hand, based on the preview image, the first key point information and first posture information of the target vehicle can be determined, which can reduce the problem of low matching accuracy of the target template image due to partial occlusion of key points of the vehicle caused by the shooting angle when matching the preset template image. At the same time, by comparing the area ratio of the vehicle in the image, the similarity of the shooting distance of the vehicle can be determined, and the preset template image can be further filtered to improve the accuracy of template matching, thereby improving the accuracy of the target template image.

[0266] On the other hand, when the target template image is determined, alignment prompts can be generated so that users can adjust the shooting angle of the electronic device based on the alignment prompts, thereby ensuring the accuracy of the user's composition and improving the user's photography experience.

[0267] Figure 5 This is a block diagram illustrating a photographing device according to an exemplary embodiment, such as... Figure 5 As shown, the photographing device 500 includes:

[0268] The information determination module 501 is configured to determine the first key point information and the first posture information of the target vehicle based on the preview image acquired for the target vehicle when the electronic device enters the shooting preview stage; wherein, the key point information is used to indicate the spatial coordinates of various components of the vehicle.

[0269] The template determination module 502 is configured to determine a target template image from a preset template image based on the first key point information and the first posture information; wherein the preset template image includes a template vehicle;

[0270] The prompting module 503 is configured to display alignment prompts on the preview image based on the position information of the template vehicle in the target template image; wherein the alignment prompts are used to indicate the adjustment of the shooting angle of the electronic device.

[0271] In some embodiments, the template determining module 502 includes:

[0272] The first submodule is configured to determine the second key point information of the template vehicle and the second posture information of the template vehicle in each of the preset template images;

[0273] The second submodule is configured to determine the first matching degree between the target vehicle and the template vehicle for each preset template image based on the first key point information, the first posture information, the second key point information, and the second posture information.

[0274] The third submodule is configured to determine the target template image from the preset template images based on each of the first matching degrees.

[0275] In some embodiments, the second submodule is specifically configured as follows:

[0276] Determine the first similarity between the first key point information and the second key point information;

[0277] Based on the angle information indicated by the first posture information, a first target point is determined from the first key point information, and based on the angle information indicated by the second posture information, a second target point is determined from the second key point information; wherein, the target point information is used to indicate the spatial coordinates of key components of the vehicle displayed in the image;

[0278] Determine the second similarity between the first target point information and the second target point information;

[0279] The first matching degree is determined based on the first similarity and the second similarity.

[0280] In some embodiments, the second submodule includes:

[0281] The selection module is configured to select third target point information that matches each of the first target point information from each of the second target point information.

[0282] The vector determination module is configured to determine first vector information associated with the first target point information and second vector information associated with the third target point information; wherein the vector information is used to characterize the attitude information of the vehicle.

[0283] The first similarity determination module is configured to determine a third similarity for each first target point information and the matched third target point information, based on the first vector information and the second vector information;

[0284] The second similarity determination module is configured to determine the second similarity based on each of the third similarities.

[0285] In some embodiments, the first similarity determination module is specifically configured as follows:

[0286] The cosine similarity between the first vector information and the second vector information is determined as the third similarity.

[0287] The determination of the second similarity based on each of the third similarities includes:

[0288] The second similarity is determined based on the average value corresponding to each of the cosine similarities.

[0289] In some embodiments, the device 500 further includes:

[0290] The proportion determination module is configured to determine the first proportion of the target vehicle in the preview image, and the second proportion of each template vehicle in the corresponding preset template image;

[0291] The processing module is configured to determine a second matching degree between the target vehicle and the template vehicle for each preset template image, based on the first proportion and the second proportion; wherein the second matching degree characterizes the similarity between the shooting distance of the target vehicle and the shooting distance of the template vehicle.

[0292] The third submodule is also configured as follows:

[0293] The target template image is determined from the preset template image based on each of the first matching degree and each of the second matching degree.

[0294] In some embodiments, the third submodule is further configured as follows:

[0295] For each of the preset template images, the first matching degree and the second matching degree are weighted based on preset weight values ​​to determine the target matching degree;

[0296] The preset template image corresponding to the highest target matching degree among all the target matching degrees is determined as the target template image.

[0297] In some embodiments, the information determination module 501 is specifically configured as follows:

[0298] Extract the image region containing the target vehicle from the preview image;

[0299] The size of the image region is adjusted to a preset size to obtain the image to be processed;

[0300] The image to be processed is input into the target recognition model to obtain the first key point information and the first pose information;

[0301] The target recognition model is used to identify key point information and attitude information of vehicles in the image.

[0302] In some embodiments, the device 500 further includes:

[0303] The first acquisition module is configured to acquire sample images taken of a preset vehicle, third key point information of the preset vehicle, third posture information of the preset vehicle, and a target heat map generated based on the third key point information.

[0304] The second acquisition module is configured to input the sample image into the initial recognition model to be trained, and the initial recognition model outputs predicted key point information, predicted pose information and predicted heat map.

[0305] The update module is configured to update the network parameters of the initial recognition model based on the difference information between the predicted key point information and the third key point information, the difference information between the predicted pose information and the third pose information, and the difference information between the predicted heatmap and the target heatmap.

[0306] The generation module is configured to obtain the target recognition model when the updated network parameters meet the preset convergence conditions.

[0307] In some embodiments, the second acquisition module is specifically configured as follows:

[0308] The feature extraction network of the initial recognition model is used to extract the feature information of the preset vehicle;

[0309] The feature information is identified using the generative network of the initial recognition model to obtain the predicted heatmap; wherein, the generative network is used to determine the distribution information of key point information of the vehicle.

[0310] The feature information is identified using the recognition network of the initial recognition model to obtain the predicted key point information;

[0311] The feature information is identified using the classification network of the initial recognition model to obtain the predicted pose information.

[0312] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0313] Figure 6 This is a structural block diagram illustrating an electronic device 600 according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0314] Reference Figure 6 The electronic device 600 may include one or more of the following components: processing component 602, memory 604, power supply component 606, multimedia component 608, audio component 610, input / output (I / O) interface 612, sensor component 614, and communication component 616.

[0315] Processing component 602 typically controls the overall operation of electronic device 600, such as operations associated with at least one of display, telephone call, data communication, camera operation, and recording operation. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0316] Memory 604 is configured to store various types of data to support the operation of electronic device 600. Examples of such data include at least one of the following: instructions for any application or method operating on electronic device 600, contact data, phonebook data, messages, pictures, and videos. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0317] Power supply component 606 provides power to various components of electronic device 600. Power supply component 606 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 600.

[0318] Multimedia component 608 includes a screen that provides an output interface between electronic device 600 and user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When electronic device 600 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0319] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when electronic device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0320] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, and buttons. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0321] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of electronic device 600. For example, sensor assembly 614 may detect the on / off state of electronic device 600, the relative positioning of components such as the display and keypad of electronic device 600, changes in position of electronic device 600 or one of its components, the presence or absence of user contact with electronic device 600, orientation or acceleration / deceleration of electronic device 600, and temperature changes of electronic device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetometer, a pressure sensor, and a temperature sensor.

[0322] Communication component 616 is configured to facilitate wired or wireless communication between electronic device 600 and other devices. Electronic device 600 can access wireless networks based on communication standards, such as Wi-Fi, 4G, 5G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, UWB technology, Bluetooth (BT) technology, and other technologies.

[0323] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0324] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including executable instructions or a computer program, which can be executed by a processor 620 of an electronic device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0325] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the above-described photographing methods of the present disclosure embodiments. For example, the photographing method includes:

[0326] When the electronic device enters the shooting preview stage, based on the preview image acquired for the target vehicle, the first key point information and the first posture information of the target vehicle are determined; wherein, the key point information is used to indicate the spatial coordinates of various components of the vehicle.

[0327] Based on the first key point information and the first posture information, a target template image is determined from a preset template image; wherein, the preset template image includes a template vehicle;

[0328] Based on the position information of the template vehicle in the target template image, alignment prompts are displayed on the preview image; wherein, the alignment prompts are used to indicate the shooting angle of the electronic device.

[0329] This disclosure provides a computer program product comprising a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the computer device to perform any of the above-described photographing methods of this disclosure.

[0330] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0331] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for taking photos, characterized in that, The method includes: When the electronic device enters the shooting preview stage, based on the preview image acquired for the target vehicle, the first key point information and the first posture information of the target vehicle are determined; wherein, the key point information is used to indicate the spatial coordinates of various components of the vehicle. Based on the first key point information and the first posture information, a target template image is determined from a preset template image; wherein, the preset template image includes a template vehicle; Based on the position information of the template vehicle in the target template image, alignment prompts are displayed on the preview image; wherein, the alignment prompts are used to indicate the shooting angle of the electronic device.

2. The method according to claim 1, characterized in that, The step of determining the target template image from the preset template image based on the first key point information and the first pose information includes: Determine the second key point information and the second posture information of the template vehicle in each of the preset template images; For each of the preset template images, a first matching degree between the target vehicle and the template vehicle is determined based on the first key point information, the first posture information, the second key point information, and the second posture information. Based on each of the first matching degrees, the target template image is determined from the preset template images.

3. The method according to claim 2, characterized in that, The step of determining the first matching degree between the target vehicle and the template vehicle based on the first key point information, the first posture information, the second key point information, and the second posture information includes: Determine the first similarity between the first key point information and the second key point information; Based on the angle information indicated by the first posture information, a first target point is determined from the first key point information, and based on the angle information indicated by the second posture information, a second target point is determined from the second key point information; wherein, the target point information is used to indicate the spatial coordinates of key components of the vehicle displayed in the image; Determine the second similarity between the first target point information and the second target point information; The first matching degree is determined based on the first similarity and the second similarity.

4. The method according to claim 3, characterized in that, Determining the second similarity between the first target point information and the second target point information includes: From each of the second target point information, select the third target point information that matches each of the first target point information; A first vector information associated with the first target point information and a second vector information associated with the third target point information are determined; wherein the vector information is used to characterize the attitude information of the vehicle; For each first target point information and the matched third target point information, a third similarity is determined based on the first vector information and the second vector information; The second similarity is determined based on each of the third similarities.

5. The method according to claim 4, characterized in that, The step of determining the third similarity based on the first vector information and the second vector information includes: The cosine similarity between the first vector information and the second vector information is determined as the third similarity. The determination of the second similarity based on each of the third similarities includes: The second similarity is determined based on the average value corresponding to each of the cosine similarities.

6. The method according to claim 2, characterized in that, The method further includes: Determine the first proportion of the target vehicle in the preview image, and the second proportion of each template vehicle in the corresponding preset template image; For each of the preset template images, a second matching degree between the target vehicle and the template vehicle is determined based on the first proportion and the second proportion; wherein, the second matching degree characterizes the similarity between the shooting distance of the target vehicle and the shooting distance of the template vehicle; Determining the target template image from the preset template images based on each of the first matching degrees includes: The target template image is determined from the preset template image based on each of the first matching degree and each of the second matching degree.

7. The method according to claim 6, characterized in that, The step of determining the target template image from the preset template image based on each of the first matching degrees and each of the second matching degrees includes: For each of the preset template images, the first matching degree and the second matching degree are weighted based on preset weight values ​​to determine the target matching degree; The preset template image corresponding to the highest target matching degree among all the target matching degrees is determined as the target template image.

8. The method according to any one of claims 1 to 7, characterized in that, The step of determining the first key point information and the first attitude information of the target vehicle based on the preview image acquired for the target vehicle includes: Extract the image region containing the target vehicle from the preview image; The size of the image region is adjusted to a preset size to obtain the image to be processed; The image to be processed is input into the target recognition model to obtain the first key point information and the first pose information; The target recognition model is used to identify key point information and attitude information of vehicles in the image.

9. The method according to claim 8, characterized in that, The method further includes: Acquire sample images of a preset vehicle, third key point information of the preset vehicle, third attitude information of the preset vehicle, and a target heat map generated based on the third key point information. The sample image is input into the initial recognition model to be trained, and the initial recognition model outputs predicted key point information, predicted pose information, and predicted heatmap. Based on the differences between the predicted key point information and the third key point information, the differences between the predicted pose information and the third pose information, and the differences between the predicted heatmap and the target heatmap, the network parameters of the initial recognition model are updated. The target recognition model is obtained when the updated network parameters meet the preset convergence conditions.

10. The method according to claim 9, characterized in that, The process involves inputting the sample image into an initial recognition model to be trained, wherein the initial recognition model outputs predicted keypoint information, predicted pose information, and a predicted heatmap, including: The feature extraction network of the initial recognition model is used to extract the feature information of the preset vehicle; The feature information is identified using the generative network of the initial recognition model to obtain the predicted heatmap; wherein, the generative network is used to determine the distribution information of key point information of the vehicle. The feature information is identified using the recognition network of the initial recognition model to obtain the predicted key point information; The feature information is identified using the classification network of the initial recognition model to obtain the predicted pose information.

11. A photographing device, characterized in that, The device includes: The information determination module is configured to determine the first key point information and the first posture information of the target vehicle based on the preview image acquired for the target vehicle when the electronic device enters the shooting preview stage; wherein, the key point information is used to indicate the spatial coordinates of various components of the vehicle. The template determination module is configured to determine a target template image from a preset template image based on the first key point information and the first posture information; wherein the preset template image includes a template vehicle; The prompting module is configured to display alignment prompts on the preview image based on the position information of the template vehicle in the target template image; wherein the alignment prompts are used to indicate the adjustment of the shooting angle of the electronic device.

12. An electronic device, characterized in that, include: processor; Memory used to store computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing a computer program or instructions, characterized in that, When the computer program or instructions in the storage medium are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 10.

Images