Vehicle trunk opening method and device, electronic equipment, medium and vehicle

Abstract

The invention relates to a vehicle trunk opening method and device, electronic equipment, a medium and a vehicle, in particular to the technical field of vehicles. The method comprises the following steps: acquiring an image sequence under the condition that a user is detected to be in a preset range; inputting the image sequence into a human body key point detection model to obtain a leg key point coordinate sequence output by the human body key point detection model; inputting the leg key point coordinate sequence into an action recognition model to obtain a predicted action output by the action recognition model; under the condition that the predicted action is matched with the preset action, whether the foot key point coordinate sequence tends to move towards a vehicle trunk or not is judged; and under the condition that the foot key point coordinate sequence tends to move towards the vehicle trunk, the vehicle trunk is controlled to be opened. According to the embodiment of the invention, the opening of the vehicle trunk can be accurately and safely controlled.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a method, device, electronic equipment, medium and vehicle for opening a vehicle trunk. Background Art

[0002] At present, the ways to open the trunk of a vehicle include manual switch, voice control, kicking under the rear bumper, and image recognition. Manual switch is very inconvenient when the user is holding an object. Voice control is easily disturbed by noise and has poor stability.

[0003] The "kick" method under the rear bumper is to install a capacitive sensor under the rear bumper. The user makes a specified action with his foot, and the capacitive sensor converts the user's action signal into a high-low level signal, and then judges the high-low level signal to determine whether the user has made a preset action. After the verification, the trunk is opened, but the user's operating range under this scheme is limited to the rear bumper position. Therefore, a convenient and efficient vehicle trunk opening method is urgently needed. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method, device, electronic device, medium and vehicle for opening a vehicle trunk, which can conveniently, efficiently and safely control the opening of the vehicle trunk.

[0005] In order to achieve the above objectives, the technical solutions provided by the embodiments of the present disclosure are as follows:

[0006] In a first aspect, the present disclosure provides a method for opening a vehicle trunk, comprising:

[0007] When it is detected that the user is within the preset range, an image sequence is acquired, wherein the image sequence includes multiple frames of images acquired by the target vehicle-mounted camera at different times;

[0008] Inputting the image sequence into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model, wherein the leg key point coordinate sequence includes a foot key point coordinate sequence for indicating a change in a foot position;

[0009] Input the leg key point coordinate sequence into the action recognition model to obtain the predicted action output by the action recognition model;

[0010] When the predicted action matches the preset action, determine whether the foot key point coordinate sequence has a tendency to move toward the vehicle trunk;

[0011] When the coordinate sequence of the key points of the foot has a tendency to move toward the trunk of the vehicle, the trunk of the vehicle is controlled to open.

[0012] As an optional implementation of the embodiment of the present disclosure, when the predicted action matches the preset action, determining whether the foot key point coordinate sequence has a tendency to move toward the trunk of the vehicle includes:

[0013] When the predicted action matches the preset action, converting the foot key point coordinate sequence in the image coordinate system into a target coordinate sequence corresponding to the foot key point coordinates in the global coordinate system according to the internal and external parameters of the target vehicle-mounted camera;

[0014] In the global coordinate system, based on the target coordinate sequence and the regional coordinates corresponding to the vehicle trunk, it is determined whether the target coordinate sequence has a trend of moving toward the vehicle trunk.

[0015] As an optional implementation of the embodiment of the present disclosure, when the foot key point coordinate sequence has a tendency to move toward the vehicle trunk, controlling the vehicle trunk to open includes:

[0016] In the case where the foot key point coordinate sequence has a tendency to move toward the trunk of the vehicle, determining whether the target foot key point coordinate is located in a preset area; the target foot key point is a foot key point that is later in time in the foot key point coordinate sequence arranged in time sequence;

[0017] When the target foot key point coordinates are located in the preset area, the vehicle trunk is controlled to open.

[0018] As an optional implementation of the embodiment of the present disclosure, a training method for a human key point detection model includes:

[0019] Obtain sample images and annotated images, where the annotated images include the coordinates of the human body external bounding box and the coordinates of the key points of the legs corresponding to the sample images;

[0020] Input the sample image and the annotated image into the pre-training model to obtain the predicted human body bounding box coordinates and predicted key point coordinates output by the pre-training model; the pre-training model includes an initial human body detection model and an initial key point detection model;

[0021] Based on the first loss function and the second loss function, respectively calculating the first loss function value between the predicted human body external bounding box coordinates and the human body external bounding box coordinates, and the second loss function value between the predicted key point coordinates and the leg key point coordinates;

[0022] When the first loss function value and / or the second loss function value is greater than the loss threshold, the model parameters of the pre-trained model are adjusted until the first loss function value and the second loss function value are less than or equal to the loss threshold, thereby obtaining a converged human key point detection model.

[0023] As an optional implementation of the embodiment of the present disclosure, the human key point detection model includes a human body detection model and a key point detection model. The image sequence is input into the human body key point detection model to obtain a leg key point coordinate sequence output by the human body key point detection model, including:

[0024] Input the image sequence into the human body detection model to obtain the coordinates of the human body external frame output by the human body detection model;

[0025] The image sequence is cropped according to the coordinates of the human body external frame to obtain a portrait sequence;

[0026] The portrait sequence is input into the key point detection model to obtain the leg key point coordinate sequence output by the key point detection model.

[0027] As an optional implementation of the embodiment of the present disclosure, a method for training an action recognition model includes:

[0028] Obtain the sample key point coordinate sequence and the labeling action corresponding to the sample key point sequence;

[0029] Input the sample key point coordinate sequence and the annotated action into the initial action recognition model;

[0030] The initial action recognition model is trained according to the output actions of the initial action recognition model and the labeled actions corresponding to the sample key point coordinate sequence.

[0031] In a second aspect, the present disclosure provides a vehicle trunk opening device, the device comprising:

[0032] An acquisition module is used to acquire an image sequence when it is detected that the user is within a preset range, the image sequence including multiple frames of images acquired by the target vehicle-mounted camera at different times;

[0033] A key point detection module, used for inputting an image sequence into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model, wherein the leg key point coordinate sequence includes a foot key point coordinate sequence for indicating a change in a foot position;

[0034] An action recognition module is used to input the coordinate sequence of the key points of the legs into the action recognition model to obtain the predicted action output by the action recognition model;

[0035] A judgment module, used to judge whether the coordinate sequence of the key points of the foot has a tendency to move toward the trunk of the vehicle when the predicted action matches the preset action;

[0036] The control module is used to control the opening of the vehicle trunk when the foot key point coordinate sequence has a tendency to move toward the vehicle trunk.

[0037] As an optional implementation of the embodiment of the present disclosure, the judgment module is specifically used to: when the predicted action matches the preset action, according to the internal and external parameters of the target vehicle-mounted camera, convert the foot key point coordinate sequence in the image coordinate system into the target coordinate sequence corresponding to the foot key point coordinates in the global coordinate system;

[0038] In the global coordinate system, based on the target coordinate sequence and the regional coordinates corresponding to the vehicle trunk, it is determined whether the target coordinate sequence has a trend of moving toward the vehicle trunk.

[0039] As an optional implementation of the embodiment of the present disclosure, the control module is specifically used to: when the foot key point coordinate sequence has a tendency to move toward the trunk of the vehicle, determine whether the target foot key point coordinate is located in a preset area; the target foot key point is a foot key point that is later in time in the foot key point coordinate sequence arranged in time sequence;

[0040] When the target foot key point coordinates are located in the preset area, the vehicle trunk is controlled to open.

[0041] As an optional implementation of the embodiment of the present disclosure, the device further includes: a model training module, which is used to train a human key point detection model, specifically used to:

[0042] Obtain sample images and annotated images, where the annotated images include the coordinates of the human body external bounding box and the coordinates of the key points of the legs corresponding to the sample images;

[0043] Input the sample image and the annotated image into the pre-training model to obtain the predicted human body bounding box coordinates and predicted key point coordinates output by the pre-training model; the pre-training model includes an initial human body detection model and an initial key point detection model;

[0044] Based on the first loss function and the second loss function, respectively calculating the first loss function value between the predicted human body external bounding box coordinates and the human body external bounding box coordinates, and the second loss function value between the predicted key point coordinates and the leg key point coordinates;

[0045] When the first loss function value and / or the second loss function value is greater than the loss threshold, the model parameters of the pre-trained model are adjusted until the first loss function value and the second loss function value are less than or equal to the loss threshold, thereby obtaining a converged human key point detection model.

[0046] As an optional implementation of the embodiment of the present disclosure, the human key point detection model includes a human detection model and a key point detection model;

[0047] The key point detection module is specifically used to: input the image sequence into the human body detection model to obtain the coordinates of the human body external frame output by the human body detection model;

[0048] The image sequence is cropped according to the coordinates of the human body external frame to obtain a portrait sequence;

[0049] The portrait sequence is input into the key point detection model to obtain the leg key point coordinate sequence output by the key point detection model.

[0050] As an optional implementation of the embodiment of the present disclosure, the device further includes: a model training module, which is used to train the action recognition model, specifically used to: obtain a sample key point coordinate sequence and annotated actions corresponding to the sample key point sequence;

[0051] Input the sample key point coordinate sequence and the annotated action into the initial action recognition model;

[0052] The initial action recognition model is trained according to the output actions of the initial action recognition model and the labeled actions corresponding to the sample key point coordinate sequence.

[0053] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for opening a vehicle trunk as described in the first aspect or any one of its optional embodiments is implemented.

[0054] In a fourth aspect, the present disclosure provides a computer-readable storage medium, comprising: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for opening a vehicle trunk as described in the first aspect or any optional embodiment thereof is implemented.

[0055] In a fifth aspect, an embodiment of the present disclosure provides a vehicle, comprising: a target vehicle-mounted camera; and an opening device for a vehicle trunk as described in the second aspect or any optional embodiment thereof, or an electronic device as described in the third aspect.

[0056] In a sixth aspect, the present disclosure provides a computer program product, comprising: the computer program product comprises a computer program, and when the computer program runs on a computer, the computer implements the vehicle trunk opening method as described in the first aspect or any optional embodiment thereof.

[0057] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0058] An embodiment of the present disclosure provides a method for opening a vehicle trunk. When a user is detected to be within a preset range, the method obtains an image sequence, wherein the image sequence includes multiple frames of images acquired by a target vehicle-mounted camera at different times, and the multiple frames of images are sorted according to the acquisition time. The image sequence is then input into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model. The leg key point coordinate sequence is then input into a motion recognition model to obtain a predicted action output by the motion recognition model. When the predicted action matches the preset action, it is determined whether a foot key point coordinate sequence representing a change in foot position in the leg key point coordinate sequence has a trend of moving toward the vehicle trunk. If so, it means that the user makes the preset action and walks towards the trunk in order to open the trunk, and the vehicle trunk is controlled to be opened. Compared with the manual switch starting method, the present invention does not require the operation of the physical switch, and can realize the opening of the trunk remotely, which is very convenient; compared with the voice control starting method, the present invention utilizes image recognition technology that is less affected by the environment and has stronger stability. In addition, unlike the existing image recognition starting method, the present invention focuses on the detected leg key point coordinate sequence to accurately identify the user's leg movements in order to reduce misjudgment. When the user makes a preset action, it further determines whether the user walks towards the trunk of the vehicle. It can eliminate the interference of other irrelevant people walking back and forth near the vehicle, thereby achieving precise control of the opening of the vehicle trunk. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 A schematic flow chart of a method for opening a vehicle trunk according to an embodiment of the present disclosure;

[0062] Figure 2A A schematic diagram of a preset range provided for an embodiment of the present disclosure;

[0063] Figure 2B A schematic diagram of a target vehicle-mounted camera provided by an embodiment of the present disclosure;

[0064] Figure 3 A schematic diagram of an application of a human body detection model provided in an embodiment of the present disclosure;

[0065] Figure 4A schematic diagram of an application of a key point detection model provided in an embodiment of the present disclosure;

[0066] Figure 5 A schematic diagram of an application of the action recognition model provided by an embodiment of the present disclosure;

[0067] Figure 6 A schematic diagram of a preset area provided in an embodiment of the present disclosure;

[0068] Figure 7 It is a structural schematic diagram of a vehicle trunk opening device according to an embodiment of the present disclosure;

[0069] Figure 8 The present invention is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0070] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0071] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0072] In order to solve some or all of the technical problems existing in the related art, an embodiment of the present disclosure provides a method for opening a vehicle trunk. When detecting that a user is within a preset range, the method obtains an image sequence, the image sequence including multiple frames of images acquired by a target vehicle-mounted camera at different times, and the multiple frames of images are sorted according to the acquisition time; the image sequence is then input into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model, and then the leg key point coordinate sequence is input into a motion recognition model to obtain a predicted action output by the motion recognition model. When the predicted action matches the preset action, it is determined whether a foot key point coordinate sequence representing a change in the foot position in the leg key point coordinate sequence has a trend of moving toward the vehicle trunk. If so, it means that the user makes a preset action and walks towards the trunk to open the trunk, and the vehicle trunk is controlled to open. Compared with the manual switch starting method, the present invention does not require the operation of the physical switch, and can realize the opening of the trunk remotely, which is very convenient; compared with the voice control starting method, the present invention utilizes image recognition technology that is less affected by the environment and has stronger stability. In addition, unlike the existing image recognition starting method, the present invention focuses on the detected leg key point coordinate sequence to accurately identify the user's leg movements in order to reduce misjudgment. When the user makes a preset action, it further determines whether the user walks towards the trunk of the vehicle. It can eliminate the interference of other irrelevant people walking back and forth near the vehicle, thereby achieving precise control of the opening of the vehicle trunk.

[0073] A method for opening a vehicle trunk provided in an embodiment of the present disclosure can be implemented by an opening device or an electronic device of the vehicle trunk, and the electronic device includes but is not limited to a server, a personal computer, a laptop computer, a tablet computer, a smart phone, a vehicle-mounted device, etc. The operating system of the electronic device may include Android, a mobile operating system (iOS) developed by Apple, an operating system (Windows) developed by Microsoft Corporation of the United States, etc., and the embodiment of the present disclosure does not limit this. The electronic device can be operated alone to implement the present disclosure, or it can be connected to a network and implement the present disclosure through interactive operations with other computer devices in the network. Among them, the network in which the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN) network, etc.

[0074] It should be noted that the protection scope of the vehicle trunk opening method described in the embodiment of the present disclosure is not limited to the execution order of the steps listed in this embodiment, and all solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0075] like Figure 1 As shown, Figure 1The figure is a flow chart of a method for opening a vehicle trunk according to an embodiment of the present disclosure. The method can be executed by a vehicle trunk opening device, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 1 As shown, the method mainly includes the following steps S101 to S105:

[0076] S101. Acquire an image sequence when it is detected that a user is within a preset range.

[0077] The preset range is the pre-set sensor detectable range, such as image sensors, ultrasonic sensors, infrared sensors, etc. The preset range can be regarded as a rough recognition range. When a user approaches the vehicle and is within the preset range, the recognition process of controlling the opening of the vehicle trunk is triggered. Figure 2A As shown, Figure 2A This is a schematic diagram of a preset range provided in an embodiment of the present disclosure, where a shadow area 201 1 m away from the vehicle body is defined as the preset range.

[0078] The target vehicle-mounted camera, in the embodiment of the present disclosure, may be a surround-view fisheye camera, which is used to collect images of the front, rear, left, and right directions of the vehicle. The shooting range of the surround-view fisheye camera includes a 360° area near the vehicle. It should be emphasized that the shooting range of the surround-view fisheye camera covers the aforementioned preset range.

[0079] For example, Figure 2B As shown, Figure 2B The schematic diagram of the surround-view fisheye camera provided in the embodiment of the present disclosure includes a front-view fisheye camera, a right-view fisheye camera, a rear-view fisheye camera, and a left-view fisheye camera, which are respectively arranged at the positions 21, 22, 23, and 24 shown in the figure. The present disclosure does not limit the installation position and specific number of the surround-view fisheye camera.

[0080] The image sequence captured by the panoramic fisheye camera includes multiple frames of images captured by the same fisheye camera at different times. The panoramic fisheye camera includes at least four fisheye cameras. Accordingly, the image sequence includes multiple frames of images captured by at least four fisheye cameras at different times. The image sequence is grouped according to the fisheye cameras, and then includes at least four groups of images, each group of images is multiple frames of images captured by the corresponding fisheye camera at different times; each group of images is sorted according to the acquisition time.

[0081] In the related art, face recognition technology is also used to identify the user approaching the trunk of the vehicle, so that the authorized user can control the opening of the trunk. However, excessive use of face recognition technology can easily lead to information leakage and pose an information security risk. Therefore, in the embodiment of the present disclosure, the target vehicle-mounted camera can be installed in a preset position, which can ensure that each image collected does not contain facial information, that is, each frame of the image in the image sequence only contains other information besides the facial information.

[0082] In some embodiments, the acquisition frequency of the surround fisheye camera is set, for example, 10 frames per second, and the acquired image sequence includes multiple groups of images, each group includes multiple frames of images. Different fisheye cameras correspond to different groups. For example, an image sequence acquired by the surround fisheye camera within two seconds is obtained, and the image sequence includes four groups of images: 20 frames of images corresponding to the front fisheye camera, 20 frames of images corresponding to the right fisheye camera, 20 frames of images corresponding to the rear fisheye camera, and 20 frames of images corresponding to the left fisheye camera.

[0083] The disclosed embodiment uses a surround-view fisheye camera to capture images covering a 360° area around the vehicle. Based on the surround-view fisheye camera equipped on most existing vehicles, there is no need to additionally set a capacitive sensor on the rear bumper. The image capture and subsequent visual algorithm can effectively improve the control accuracy of the trunk opening and reduce misjudgment.

[0084] S102, inputting the image sequence into a human body key point detection model to obtain a leg key point coordinate sequence output by the human body key point detection model.

[0085] Among them, the human body key point detection model includes a human body detection model and a key point detection model connected in sequence.

[0086] The disclosed embodiment provides a training method for a human key point detection model, optionally obtaining a sample image and an annotated image, wherein the annotated image includes the coordinates of a human body external bounding box corresponding to the sample image, and the coordinates of key points corresponding to a leg region in the sample image; the sample image and the annotated image are input into a pre-training model to obtain predicted human body external bounding box coordinates and predicted key point coordinates output by the pre-training model; the pre-training model includes an initial human body detection model and an initial key point detection model; based on a first loss function and a second loss function, a first loss function value between the predicted human body external bounding box coordinates and the human body external bounding box coordinates, and a second loss function value between the predicted key point coordinates and the key point coordinates corresponding to the leg region are calculated respectively; when the first loss function value and / or the second loss function value is greater than a loss threshold, the model parameters of the pre-training model are adjusted until the first loss function value and the second loss function value are less than or equal to the loss threshold, and a converged human body key point detection model is obtained.

[0087] Specifically, in the process of training the human key point detection model, firstly obtain sample images and annotated images, wherein the annotated images are annotated with the human body external frame coordinates and the leg key point coordinates based on the sample images, and the leg key point coordinates include: left hip key point coordinates, right hip key point coordinates, left knee key point coordinates, right knee key point coordinates, left foot key point coordinates and right foot key point coordinates, which are only illustrative here, and the present disclosure does not specifically limit the leg key point coordinates; a pre-training model will be constructed in advance, and the pre-training model includes an initial human body detection model and an initial key point detection model, and the structure of the pre-training model may include a backbone network, a shoulder network and a head network. Then, the sample image and the annotated image are input into the pre-training model as training sets, and the pre-training model performs calculation processing to identify the human body external bounding box coordinates and leg key point coordinates of the sample image, and obtain the predicted human body external bounding box coordinates and predicted key point coordinates output by the pre-training model, which are regarded as predicted values; further, the human body external bounding box coordinates corresponding to the annotated image and the key point coordinates corresponding to the leg area are taken as true values, and the loss function values ​​between the predicted values ​​and the true values ​​are calculated based on the first loss function and the second loss function, including: calculating the first loss function value between the predicted human body external bounding box coordinates and the human body external bounding box coordinates corresponding to the annotated image based on the first loss function, and calculating the second loss function value between the predicted key point coordinates and the leg key point coordinates in the annotated image based on the second loss function, wherein the first loss function and the second loss function may be a cross entropy loss function, a contrastive learning loss function, etc., and the present disclosure does not specifically limit this. The first loss function and the second loss function may be the same or different. Then, the first loss function value and the second loss function value are compared with the loss threshold. The loss threshold is the critical value of the loss function that characterizes the convergence of the model. If it is greater than the loss threshold, it means that the model has not converged, and if it is less than or equal to the loss threshold, it means that the model has converged. When the first loss function value is greater than the loss threshold and the second loss function value is less than or equal to the loss threshold, or when the second loss function value is greater than the loss threshold and the first loss function value is less than or equal to the loss threshold, or when the first loss function value and the second loss function value are greater than the loss threshold, it means that the model has not converged, and the model parameters of the pre-trained model are adjusted until the first loss function value and the second loss function value are both less than or equal to the loss threshold, indicating that the model has converged, and then a trained human key point detection model is obtained.

[0088] In some embodiments, the human body detection model and the key point detection model included in the human body key point detection model can be trained separately and independently. The following will introduce the training process of the human body detection model and the training process of the key point detection model respectively.

[0089] (1) Human body detection model

[0090] In some embodiments, the training method of the human body detection model includes: obtaining a sample image and a first annotated image, the first annotated image including the coordinates of the human body external frame corresponding to the sample image; inputting the sample image and the first annotated image into an initial human body detection model, the initial human body detection model including a backbone network, a shoulder network and a head network; training the initial human body detection model according to the predicted human body external frame coordinates output by the initial human body detection model to obtain a converged human body detection model.

[0091] The specific steps include:

[0092] ① Obtain model training set

[0093] The sample image and the first annotated image are obtained as a model training set. The first annotated image includes the coordinates of the human body external bounding box corresponding to the sample image. It can be understood that the first annotated image annotates the human body position based on the sample image and annotates the human body external bounding box corresponding to the human body. Optionally, the sample image includes positive sample images and negative sample images. The positive sample image is an image containing a human body, and the negative sample image is an image not containing a human body. Combining positive and negative sample images for model training can improve the robustness and accuracy of the model.

[0094] ②Build the initial model

[0095] The constructed initial human detection model includes a backbone network, a shoulder network, and a head network.

[0096] ③Model training

[0097] Input the sample image and the first annotated image into the initial human body detection model, obtain the predicted human body external frame coordinates output by the initial human body detection model, compare the predicted human body external frame coordinates with the real human body external frame coordinates (the human body external frame coordinates corresponding to the sample image), and calculate the first loss function value corresponding to the two based on the first loss function. If the first loss function value is greater than the loss threshold, adjust the model parameters of the initial human body detection model, and then perform a new round of training until the first loss function value is less than or equal to the loss threshold, and obtain a converged human body detection model. The first loss function includes but is not limited to the cross entropy loss function. The loss threshold is the critical value of the loss function that characterizes the convergence of the model. If it is greater than the loss threshold, it means that the model has not converged, and if it is less than or equal to the loss threshold, it means that the model has converged.

[0098] (2) Key point detection model

[0099] In some embodiments, the training method of the key point detection model includes: obtaining a sample image and a second annotated image, the second annotated image including the key point coordinates corresponding to the human body area in the sample image; inputting the sample image and the second annotated image into the initial key point detection model; and training the initial key point detection model according to the predicted key point coordinates output by the initial key point detection model to obtain a converged key point detection model.

[0100] The specific steps include:

[0101] ① Obtain model training set

[0102] A sample image and a second annotated image are obtained as a model training set, wherein the second annotated image includes key point coordinates corresponding to a human body region in the sample image.

[0103] In the process of obtaining the second annotated image, the key point circumscribed frame corresponding to the human body area in the sample image can be marked first, and then the sample image can be cropped according to the key point circumscribed frame to obtain the corresponding human body image, and the leg key points can be further marked in the human body image to obtain the second annotated image.

[0104] ②Build the initial model

[0105] The constructed initial key point detection model includes a backbone network, a shoulder network and a head network. It should be noted that although the structure of the initial key point detection model is the same as that of the initial human body detection model, the relevant parameters of each network are different. The initial human body detection model and the initial key point detection model can be constructed with reference to the prior art, which will not be elaborated in this disclosure.

[0106] ③Model training

[0107] The sample image and the second annotated image are input into the initial key point detection model to obtain the predicted key point coordinates output by the initial key point detection model, and the predicted key point coordinates are compared with the true key point coordinates (the leg key point coordinates corresponding to the sample image). The second loss function values ​​corresponding to the two are calculated based on the second loss function. When the second loss function value is greater than the loss threshold, the model parameters of the initial key point detection model are adjusted, and then a new round of training is performed until the second loss function value is less than or equal to the loss threshold, so as to obtain a converged key point detection model for identifying the leg key points in the image.

[0108] Based on the above embodiments, a converged human body detection model and a key point detection model are trained, and when the model is applied to identify key points, an image sequence is input into the human body key point detection model to obtain a leg key point coordinate sequence output by the human body key point detection model, including: inputting the image sequence into the human body detection model to obtain the human body external frame coordinates output by the human body detection model; cropping the image sequence according to the human body external frame coordinates to obtain a portrait sequence; inputting the portrait sequence into the key point detection model to obtain a leg key point coordinate sequence output by the key point detection model.

[0109] Specifically, Figure 3 As shown, Figure 3 The application diagram of the human body detection model provided in the embodiment of the present disclosure is as follows: first, the image sequence obtained in step S101 is input into the trained converged human body detection model to obtain the coordinates of the human body external bounding box output by the human body detection model, and then the images in the image sequence are cropped according to the coordinates of the human body external bounding box to obtain a portrait sequence. Figure 4 As shown, Figure 4 This is a schematic diagram of the application of the key point detection model provided in an embodiment of the present disclosure. A portrait sequence is input into the key point detection model to obtain a leg key point coordinate sequence output by the key point detection model.

[0110] The above embodiment processes the image sequence through the human body key point detection model to obtain the leg key point coordinate sequence corresponding to the image sequence for subsequent recognition of user actions.

[0111] S103, inputting the leg key point coordinate sequence into the action recognition model to obtain the recognition result output by the action recognition model.

[0112] In some embodiments, the training method of the action recognition model includes: obtaining a sample key point coordinate sequence and an annotated action corresponding to the sample key point coordinate sequence; inputting the key point coordinate sequence and the annotated action into an initial action recognition model, the initial action recognition model including a feature extraction module, a feature fusion module and a classification module; and training the initial action recognition model according to the output action of the initial action recognition model.

[0113] Specifically, the process of training the action recognition model includes the following steps:

[0114] ① Obtain model training set

[0115] Obtain a sample key point coordinate sequence and a marked action corresponding to the sample key point coordinate sequence, wherein the marked action is to mark the corresponding action based on the sample key point coordinate sequence. Optionally, the sample key point coordinate sequence is a human leg key point coordinate sequence, including left and right hips, left and right knees, left and right ankles, and left and right feet. The marked action includes but is not limited to the kicking action.

[0116] ②Build the initial model

[0117] The initial action recognition model as the initial model includes a feature extraction module, a feature fusion module and a classification module. The feature extraction module uses a convolutional neural network (CNN), the feature fusion module uses a machine translation (transformer) structure to fuse and associate features in time series, and the classification module uses a fully connected layer for binary classification.

[0118] ③Model training

[0119] The sample key point coordinate sequence and the labeled action corresponding to the sample key point coordinate sequence are input into the initial action recognition model, and the predicted action output by the initial action recognition model is identified. The classification module determines whether the predicted action matches the labeled action, and the loss function value of the predicted action and the labeled action is calculated based on the preset loss function. When the loss function value is greater than the preset threshold, the model parameters of the initial action recognition model are adjusted, and then a new round of training is performed until the loss function value is less than or equal to the preset loss function value, and a converged action recognition model is obtained.

[0120] like Figure 5 As shown, Figure 5 This is a schematic diagram of the application of the action recognition model provided in the embodiment of the present disclosure. After the converged action recognition model is obtained through training using the above training method, the leg key point coordinate sequence obtained in step S103 is input into the action recognition model to obtain the predicted action output by the action recognition model.

[0121] In order to meet the user's habit of continuing to use the kicking method to open the trunk, the present invention uses the input action recognition model obtained by recognition to identify whether the user's leg action is a kicking action.

[0122] S104: When the predicted action matches the preset action, determine whether the foot key point coordinate sequence has a tendency to move toward the vehicle trunk.

[0123] The preset action may be a kicking action. The foot key point coordinate sequence includes a left foot key point and a right foot key point whose positions change in time sequence, and can represent the change trend of the user's foot position, including the change direction and distance.

[0124] In some embodiments, when the predicted action matches the preset action, it is determined whether the foot key point coordinate sequence has a tendency to start toward the trunk of the vehicle, including: when the predicted action matches the preset action, according to the internal and external parameters of the fisheye camera, the foot key point coordinate sequence in the image coordinate system is converted into a corresponding target coordinate sequence in the global coordinate system, the target coordinate sequence includes the foot key point coordinates in the global coordinate system; in the global coordinate system, based on the target coordinate sequence and the area coordinates corresponding to the vehicle trunk, it is determined whether the target sequence has a tendency to move toward the trunk of the vehicle.

[0125] The internal and external parameters of the fisheye camera are the camera internal parameters and camera external parameters of the fisheye camera, which are used for coordinate system conversion. The area corresponding to the vehicle trunk is the area occupied by the vehicle trunk in the global coordinate system, or it can be the area within 1m from the vehicle trunk. The preset area can be the area 1m away from the left rear side of the vehicle, the right rear side of the vehicle, and the rear side of the vehicle, such as Figure 6 As shown, Figure 6 A schematic diagram of the area corresponding to the vehicle trunk provided in an embodiment of the present disclosure. The shadow area 1m away from the left rear door, 1m away from the right rear door, and 1m away from the rear bumper is defined as the area 601 corresponding to the vehicle trunk.

[0126] Specifically, when the predicted action matches the kicking action, it means that the user is photographed making a kicking action and preliminarily determines that he may want to open the trunk. When determining whether the user really wants to open the trunk, first, according to the internal and external parameters of the fisheye camera, the foot key points in the image coordinate system are projected and transformed into the global coordinate system to perform perspective transformation to determine the target coordinates of the foot key points in the bird's-eye view, and obtain the target coordinate sequence corresponding to the foot key point coordinate sequence, which can reflect the changes of the user's feet over time in the bird's-eye view. It can be understood that the target coordinate sequence is the coordinates of the foot key points arranged in chronological order from small to large, which can reflect the relative position between the user and the vehicle trunk. In the global coordinate system, according to the target sequence and the area corresponding to the vehicle trunk, it is judged whether the target coordinate sequence has a trend toward the vehicle trunk, that is, in the bird's-eye view, it is judged whether the vector corresponding to the foot key point coordinates in the target coordinate sequence points to the area corresponding to the vehicle trunk, and whether the distance is reduced.

[0127] In the above-mentioned embodiment, when it is recognized through the collected image sequence that the user has made a preset action, in order to avoid misjudgment, it is further determined whether the coordinate sequence of the key points of the footsteps has a trend of moving towards the trunk of the vehicle, thereby controlling the opening of the vehicle trunk when the user makes the preset action and walks towards the trunk. Compared with the related technology of excluding irrelevant people from approaching the trunk of the vehicle through face recognition to open the trunk, there is no need to process the relatively private user facial information, and the user's privacy can be protected.

[0128] S105 . When the foot key point coordinate sequence has a tendency to move toward the vehicle trunk, control the vehicle trunk to open.

[0129] When the coordinate sequence of the key points of the foot shows a trend of moving toward the trunk of the vehicle, it means that the user is walking towards the trunk of the vehicle. Combined with the above steps, it can be recognized that the user makes a preset action, so it can be accurately determined that the user wants to open the trunk, thereby satisfying the user's expectation to control the opening of the trunk of the vehicle.

[0130] In some embodiments, when there is a tendency for the foot key point coordinate sequence to move toward the trunk of the vehicle, it is determined whether the target foot key point coordinate is located in a preset area, and the target foot key point is one or more foot key points with a larger time sequence in the foot key point coordinate sequence arranged in time sequence; when the target foot key point coordinate is located in the area corresponding to the vehicle, the trunk of the vehicle is controlled to open.

[0131] The preset area may be a predefined usable range of the trunk, and the area of ​​the preset area is smaller than or equal to the area corresponding to the trunk of the vehicle. It is understood that when a user enters the area, the trunk of the vehicle is controlled to be opened in response to the user's operation. For example, when a user enters the area corresponding to the vehicle, the trunk of the vehicle is controlled to be opened in response to a recognized kicking action.

[0132] Specifically, in the case where there is a trend of the foot key point coordinate sequence moving toward the vehicle trunk, in order to accurately determine whether the user has the intention to open the vehicle trunk, the present disclosure further determines the target foot key point coordinates, the target key point coordinates can be the last foot key point coordinates in the foot key point coordinate sequence, the foot key point coordinate sequence is arranged according to the time size, and the number of the last foot key points can be one or more; it is determined whether the last foot key point coordinates are in a preset area to identify whether the user walks toward the vehicle trunk and enters the operable area of ​​the vehicle trunk. If the target key point coordinates (such as the last foot key point coordinates) are in the preset area, it means that the user enters the operable area of ​​the vehicle trunk and expects to open the trunk. It can be understood that the user makes a kicking action and walks toward the vehicle trunk. In this process, he gradually approaches the vehicle and stands near the trunk, thereby meeting the user's expectations and controlling the vehicle trunk to open. If the target key point coordinates are not in the preset area, it may be that other people pass by and will not control the vehicle trunk to open, reduce misjudgment, and improve the control accuracy of the vehicle trunk opening.

[0133] In addition, when there is no trend of moving toward the trunk of the vehicle in the coordinate sequence of the key points of the footsteps, it may be that the user has unintentionally made a preset action before or after getting on the vehicle. In this case, the recognition result will not be processed and the opening of the trunk of the vehicle will not be controlled.

[0134] In summary, the present disclosure implements a method for opening a vehicle trunk. The method firstly acquires an image sequence when detecting that a user is within a preset range. The image sequence includes multiple frames of images acquired by the same fisheye camera at different times. The multiple frames of images are sorted according to the acquisition time. Each frame of the image includes other information except face information, that is, the image sequence does not involve sensitive face information; then the image sequence is input into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model, and then the leg key point coordinate sequence is input into an action recognition model to obtain an action recognition model output. The predicted action is determined. When the predicted action matches the preset action, it is determined whether the foot key points representing the foot position change in the leg key point coordinate sequence have a trend of moving toward the vehicle trunk. If so, it means that the user makes the preset action and walks towards the vehicle trunk to open the trunk, then the vehicle trunk is controlled to open, thereby accurately identifying the user action through the leg key point coordinate sequence. When the user makes the preset action, it is further determined whether the user walks towards the vehicle trunk, so as to eliminate the interference of other irrelevant people walking back and forth near the vehicle, reduce misjudgment, and achieve accurate control of the opening of the vehicle trunk.

[0135] like Figure 7 As shown, Figure 7 A vehicle trunk opening device provided in an embodiment of the present disclosure includes:

[0136] An acquisition module 701 is used to acquire an image sequence when it is detected that the user is within a preset range, where the image sequence includes multiple frames of images acquired by the target vehicle-mounted camera at different times;

[0137] The key point detection module 702 is used to input the image sequence into the human key point detection model to obtain the leg key point coordinate sequence output by the human key point detection model, wherein the leg key point coordinate sequence includes a foot key point coordinate sequence for indicating the change of the foot position;

[0138] Action recognition module 703, used to input the leg key point coordinate sequence into the action recognition model to obtain the predicted action output by the action recognition model;

[0139] A judgment module 704 is used to judge whether the foot key point coordinate sequence has a tendency to move toward the vehicle trunk when the predicted action matches the preset action;

[0140] The control module 705 is used to control the opening of the vehicle trunk when the foot key point coordinate sequence has a tendency to move toward the vehicle trunk.

[0141] As an optional implementation of the embodiment of the present disclosure, the judgment module 704 is specifically used to: when the predicted action matches the preset action, convert the foot key point coordinate sequence in the image coordinate system into a corresponding target coordinate sequence in the global coordinate system according to the internal and external parameters of the target vehicle-mounted camera, where the target coordinate sequence includes the foot key point coordinates in the global coordinate system;

[0142] In the global coordinate system, based on the target coordinate sequence and the regional coordinates corresponding to the vehicle trunk, it is determined whether the target coordinate sequence has a trend of moving toward the vehicle trunk.

[0143] As an optional implementation of the embodiment of the present disclosure, the control module 705 is specifically used to: when the foot key point coordinate sequence has a tendency to move toward the trunk of the vehicle, determine whether the target foot key point coordinate is located in a preset area; the target foot key point is one or more foot key points with a larger time sequence in the foot key point coordinate sequence arranged in time sequence;

[0144] When the target foot key point coordinates are located in the preset area, the vehicle trunk is controlled to open.

[0145] As an optional implementation of the embodiment of the present disclosure, the device further includes: a model training module, which is used to train a human key point detection model, specifically used to:

[0146] Obtain sample images and annotated images, where the annotated images include the coordinates of the human body external bounding box and the coordinates of the key points of the legs corresponding to the sample images;

[0147] Input the sample image and the annotated image into the pre-training model to obtain the predicted human body bounding box coordinates and predicted key point coordinates output by the pre-training model; the pre-training model includes an initial human body detection model and an initial key point detection model;

[0148] Based on the first loss function and the second loss function, respectively calculating the first loss function value between the predicted human body external bounding box coordinates and the human body external bounding box coordinates, and the second loss function value between the predicted key point coordinates and the leg key point coordinates;

[0149] When the first loss function value and / or the second loss function value is greater than the loss threshold, the model parameters of the pre-trained model are adjusted until the first loss function value and the second loss function value are less than or equal to the loss threshold, thereby obtaining a converged human key point detection model.

[0150] As an optional implementation of the embodiment of the present disclosure, the human key point detection model includes a human detection model and a key point detection model;

[0151] The key point detection module 702 is specifically used to: input the image sequence into the human body detection model to obtain the coordinates of the human body external frame output by the human body detection model;

[0152] The image sequence is cropped according to the coordinates of the human body external frame to obtain a portrait sequence;

[0153] The portrait sequence is input into the key point detection model to obtain the leg key point coordinate sequence output by the key point detection model.

[0154] As an optional implementation of the embodiment of the present disclosure, the device further includes: a model training module, which is used to train the action recognition model, specifically used to: obtain a sample key point coordinate sequence and annotated actions corresponding to the sample key point sequence;

[0155] Input the sample key point coordinate sequence and the annotated action into the initial action recognition model;

[0156] The initial action recognition model is trained according to the output actions of the initial action recognition model and the labeled actions corresponding to the sample key point coordinate sequence.

[0157] like Figure 8 As shown, Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, the electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, each process of the vehicle trunk opening method in the above method embodiment is implemented. And the same technical effect can be achieved, so it will not be described here to avoid repetition.

[0158] An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the method for opening a vehicle trunk in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0159] The computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0160] An embodiment of the present disclosure provides a vehicle, comprising: a fisheye camera; and, as described above, an opening device for a vehicle trunk, or as described above, an electronic device.

[0161] An embodiment of the present disclosure provides a computer program product, which stores a computer program. When the computer program is executed by a processor, each process of the method for opening the trunk of a vehicle in the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0162] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0163] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0164] In the present disclosure, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0165] In the present disclosure, memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0166] In the present disclosure, computer-readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0167] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0168] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for opening a vehicle trunk, characterized in that: include: When it is detected that the user is within the preset range, an image sequence is acquired, wherein the image sequence includes multiple frames of images acquired by the target vehicle-mounted camera at different times; Inputting the image sequence into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model, wherein the leg key point coordinate sequence includes a foot key point coordinate sequence for indicating a change in a foot position; Inputting the leg key point coordinate sequence into an action recognition model to obtain a predicted action output by the action recognition model; In the case where the predicted action matches the preset action, determining whether the foot key point coordinate sequence has a tendency to move toward the vehicle trunk; When the foot key point coordinate sequence has a tendency to move toward the vehicle trunk, the vehicle trunk is controlled to open.

2. The method according to claim 1, characterized in that When the predicted action matches the preset action, determining whether the foot key point coordinate sequence has a tendency to move toward the vehicle trunk includes: When the predicted action matches the preset action, converting the foot key point coordinate sequence in the image coordinate system into a target coordinate sequence corresponding to the foot key point coordinates in the global coordinate system according to the internal and external parameters of the target vehicle-mounted camera; In the global coordinate system, it is determined whether the target coordinate sequence has a trend of moving toward the vehicle trunk based on the target coordinate sequence and the regional coordinates corresponding to the vehicle trunk.

3. The method according to claim 1 or 2, characterized in that: When the foot key point coordinate sequence has a tendency to move toward the vehicle trunk, controlling the vehicle trunk to open includes: In the case where the foot key point coordinate sequence has a tendency to move toward the vehicle trunk, determining whether the target foot key point coordinate is located in a preset area; the target foot key point is a foot key point that is later in time in the foot key point coordinate sequence arranged in time sequence; When the target foot key point coordinates are located in a preset area, the vehicle trunk is controlled to open.

4. The method according to claim 1, characterized in that: The training method of the human key point detection model comprises: Acquire a sample image and an annotated image, wherein the annotated image includes coordinates of a human body external bounding box and coordinates of key points of a leg corresponding to the sample image; Inputting the sample image and the annotated image into a pre-training model to obtain the predicted human body bounding box coordinates and predicted key point coordinates output by the pre-training model; the pre-training model includes an initial human body detection model and an initial key point detection model; Based on the first loss function and the second loss function, respectively calculating the first loss function value between the predicted human body external bounding box coordinates and the human body external bounding box coordinates, and the second loss function value between the predicted key point coordinates and the leg key point coordinates; When the first loss function value and / or the second loss function value is greater than the loss threshold, the model parameters of the pre-trained model are adjusted until the first loss function value and the second loss function value are less than or equal to the loss threshold, thereby obtaining a converged human key point detection model.

5. The method according to claim 1, characterized in that The human body key point detection model includes a human body detection model and a key point detection model, and the step of inputting the image sequence into the human body key point detection model to obtain a leg key point coordinate sequence output by the human body key point detection model includes: Inputting the image sequence into the human body detection model to obtain the coordinates of the human body external bounding box output by the human body detection model; Cropping the image sequence according to the coordinates of the human body external frame to obtain a portrait sequence; The portrait sequence is input into the key point detection model to obtain a leg key point coordinate sequence output by the key point detection model.

6. The method according to claim 1, characterized in that The training method of the action recognition model comprises: Obtaining a sample key point coordinate sequence and a labeling action corresponding to the sample key point sequence; Inputting the sample key point coordinate sequence and the marked action into an initial action recognition model; The initial action recognition model is trained according to the output action of the initial action recognition model and the labeled action corresponding to the sample key point coordinate sequence.

7. A vehicle trunk opening device, characterized in that: include: An acquisition module, configured to acquire an image sequence when detecting that the user is within a preset range, wherein the image sequence includes multiple frames of images acquired by the target vehicle-mounted camera at different times; A key point detection module, used for inputting the image sequence into a human key point detection model to obtain a leg key point coordinate sequence output by the human key point detection model, wherein the leg key point coordinate sequence includes a foot key point coordinate sequence for indicating a change in a foot position; An action recognition module, used for inputting the leg key point coordinate sequence into an action recognition model to obtain a predicted action output by the action recognition model; A judgment module, used for judging whether the foot key point coordinate sequence has a tendency to move toward the vehicle trunk when the predicted action matches the preset action; The control module is used to control the opening of the vehicle trunk when the foot key point coordinate sequence has a tendency to move toward the vehicle trunk.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for opening a vehicle trunk as claimed in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for opening the vehicle trunk as claimed in any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: include: Target vehicle-mounted camera; And the vehicle trunk opening device as claimed in claim 7, or the electronic device as claimed in claim 8.

Images