Suspension control methods, devices, equipment and media

Abstract

This application provides a suspension control method, device, equipment, and medium applicable to users outside a vehicle, to improve user experience and suspension response stability. The method includes: in response to receiving an interaction command, acquiring a captured image and determining key point location information of the user in the captured image; wherein the key point location information includes first location information of a reference point and second location information of an interaction target point; determining first motion parameters of the suspension based on the relative positional relationship between the reference point and the interaction target point; and controlling the suspension movement based on the first motion parameters.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a suspension control method, device, equipment and medium. Background Technology

[0002] Current technologies primarily control the suspension based on the vehicle's driving status or road feedback (sensor-collected data and / or road information simulation algorithms) while the vehicle is in motion; they mainly focus on suspension control when the user is inside the vehicle's cabin. However, with the development of vehicle control technology, the need for interaction between users and the vehicle is increasing, creating an urgent need for a suspension control method suitable for users outside the vehicle. Summary of the Invention

[0003] Therefore, it is necessary to provide a suspension control method, device, equipment, and medium to address the aforementioned technical problems, thereby improving user experience and suspension response stability.

[0004] In a first aspect, embodiments of this application provide a suspension control method, including:

[0005] In response to receiving an interaction command, the system acquires a captured image and determines the location information of key points of the user in the captured image; wherein the location information of key points includes first location information of reference points and second location information of interaction target points;

[0006] Based on the relative positional relationship between the reference point and the interactive target point, the first motion parameters of the suspension are determined;

[0007] The suspension movement is controlled based on the first motion parameter.

[0008] In one embodiment, determining the first motion parameter of the suspension based on the relative positional relationship between the reference point and the interactive target point includes:

[0009] Based on the relative positional relationship, a second motion parameter of the interactive target point is determined; wherein the interactive target point corresponds to the reference point of the suspension, and the second motion parameter includes the motion direction and / or motion distance;

[0010] The first motion parameter is determined based on the second motion parameter.

[0011] In one embodiment, the reference point includes a first reference point corresponding to the central region of the user's pose;

[0012] Determining the first motion parameter of the suspension based on the relative positional relationship between the reference point and the interactive target point includes:

[0013] The first motion parameters are determined based on the relative positional relationship between the first reference point and the interactive target point.

[0014] In one embodiment, determining the key point location information of the user in the acquired image includes:

[0015] Determine the key points and the key point location information corresponding to the key points.

[0016] In one embodiment, determining the first motion parameter of the suspension based on the relative positional relationship between the reference point and the interactive target point includes:

[0017] Based on the first location information and the second location information, the relative positional relationship between the reference point and the interactive target point is determined;

[0018] Based on the relative positional relationship, the first motion parameters of the suspension are determined.

[0019] In one embodiment, the first motion parameter includes the motion distance of the suspension;

[0020] Determining the first motion parameter of the suspension based on the relative positional relationship between the reference point and the interactive target point includes:

[0021] Obtain the first coordinates of the reference point of the suspension;

[0022] A first distance is determined based on the coordinates of the first feature point and the coordinates of the first reference point, and a second distance is determined based on the coordinates of the first reference point and the coordinates of the interactive target point;

[0023] The movement distance of the suspension is determined based on the first coordinates and the first ratio between the second distance and the first distance.

[0024] In one embodiment, the reference point includes a second reference point corresponding to the shoulder region of the user's posture;

[0025] The first feature point is obtained by processing the first reference point and the second reference point through a first preset rule.

[0026] In one embodiment, determining the first distance based on the coordinates of the first feature point and the coordinates of the first reference point, and determining the second distance based on the coordinates of the first reference point and the coordinates of the interactive target point, includes:

[0027] The first distance is determined based on the ordinate of the first feature point and the ordinate of the first reference point;

[0028] The second distance is determined based on the ordinate of the first reference point and the ordinate of the interactive target point.

[0029] Since the suspension moves in an overall upward or downward direction, its coordinate changes in the lateral or depth directions are negligible. Therefore, determining the first and second distances directly through the ordinate can effectively improve the efficiency of determining the first motion parameters and save computing power, while ensuring the stability of the suspension response and the user experience.

[0030] In one embodiment, determining the travel distance of the suspension based on the first coordinates and a first ratio between the second distance and the first distance includes:

[0031] The travel distance of the suspension is determined by multiplying the preset suspension control coefficient by the first ratio based on the ordinate of the first coordinate.

[0032] In one embodiment, determining a first detection box containing the user in the acquired image by matching a first type of detection box obtained from target detection with a second type of detection box obtained from trajectory prediction includes:

[0033] In response to the fact that the matching parameter between the first type detection box and the second type detection box in the nth frame of the acquired image is less than a preset first matching parameter threshold, the first detection box of the user in the nth frame of the acquired image is determined based on the first type detection box in the (n-1)th frame of the acquired image; where n is an integer and 2≤n≤N.

[0034] In one embodiment, the first motion parameter includes the motion direction type of the suspension;

[0035] Determining the first motion parameter of the suspension based on the relative positional relationship between the reference point and the interactive target point includes:

[0036] In response to a first difference between the ordinate of the second position information and the ordinate of the first reference point being less than a preset first target threshold, the motion direction type is determined to be a first direction type; or...

[0037] In response to the first difference being greater than or equal to the first target threshold, the motion direction type is determined to be a second direction type;

[0038] The first direction corresponding to the first direction type is opposite to the second direction corresponding to the second direction type.

[0039] In one embodiment, the motion distance in the first motion parameter is determined by the following method:

[0040] Determine the second distance, and determine the fourth distance corresponding to the type of motion direction;

[0041] The second ratio is based on the ratio of the second distance to the fourth distance; wherein the second ratio corresponds to the type of motion direction.

[0042] The movement distance is determined based on the preset suspension control coefficient, the second ratio, and the extreme height corresponding to the movement direction type.

[0043] In one embodiment, determining the fourth distance corresponding to the motion direction type includes:

[0044] In response to the motion direction type being a first direction type, the distance between the first feature point and the first reference point is determined as the fourth distance;

[0045] or,

[0046] In response to the motion direction type being a second direction type, the distance between the first reference point and the second feature point is determined as the fourth distance; wherein, the second feature point corresponds to the bottom region of the user pose.

[0047] The second feature point is obtained by processing the second reference point and the first reference point through a second preset rule.

[0048] In one embodiment, the movement distance is determined according to the following formula:

[0049] Among them, D g h is the distance of motion corresponding to the type of motion direction. max The extreme height corresponding to the motion direction type is given by k, which is a preset suspension control coefficient, D4 is the fourth distance corresponding to the motion direction type, and D2 is the second distance.

[0050] In one embodiment, determining the key point location information of the user in the acquired image includes:

[0051] In the N frames of the acquired images, a first detection box containing the user is determined by matching the first type of detection box obtained from target detection with the second type of detection box obtained from trajectory prediction; N is an integer greater than or equal to 2.

[0052] Within the first detection frame of the acquired image, the user's key points and the key point location information are determined.

[0053] In one embodiment, determining a first detection box containing the user in the acquired image by matching a first type of detection box obtained from target detection with a second type of detection box obtained from trajectory prediction includes:

[0054] Target detection is performed on the acquired images to obtain the first type of detection box; wherein, the first type of detection box includes a second detection box in the (n-1)th frame of the acquired image and a third detection box in the nth frame of the acquired image, where n is an integer and 2≤n≤N;

[0055] The trajectory of the user in the second detection box in the (n-1)th frame of the acquired image is predicted to obtain the fourth detection box in the nth frame of the acquired image;

[0056] For the nth frame of the acquired image, in response to the matching parameter between the third detection box and the fourth detection box being less than a preset second matching parameter threshold, the third detection box in the nth frame of the acquired image is corrected based on the second detection box to obtain the first detection box.

[0057] In this embodiment, the accuracy of user identification is achieved by matching the first type of detection box obtained from target detection with the second type of detection box obtained from trajectory prediction. Furthermore, when the matching parameter is less than the second matching parameter threshold, correcting the third detection box effectively avoids situations where the matching parameter is less than the second matching parameter threshold due to factors such as target detection jitter, false detection, or missed detection, leading to the mistaken assumption that the first detection box is not present in the nth frame and interrupting suspension control. This further improves the accuracy of the determined first detection box and user identification, thereby enhancing the suspension's response stability.

[0058] In one embodiment, the step of correcting the third detection box in the nth frame acquired image based on the second detection box to obtain the first detection box includes:

[0059] Based on the time stamp carried by the acquired images, the first number of acquired images preceding the nth frame acquired image are determined as the target image sequence;

[0060] In response to the acquisition image in the target image sequence including the first detection box, the third detection box in the nth frame acquisition image is corrected to obtain the first detection box.

[0061] In this embodiment, by verifying the target image sequence prior to the nth frame, it is identified whether the failure to match the third and fourth detection boxes in the nth frame is a false detection caused by factors such as jitter in target detection and / or trajectory prediction. Then, the third detection box in the nth frame is corrected, which can further improve the accuracy of the first detection box, that is, improve the accuracy of user identification. This avoids premature and incorrect termination of suspension control, or the user ending the interaction before leaving the vehicle, and still controls the suspension movement based on the first motion parameters corresponding to the first detection boxes in the previous few frames, further improving the response stability of the suspension.

[0062] In one embodiment, after predicting the user's trajectory within the second detection box in the (n-1)th frame of the acquired image to obtain the fourth detection box in the nth frame, the method further includes:

[0063] In response to the matching parameter between the third detection box and the fourth detection box being greater than or equal to the second matching parameter threshold, the third detection box of the nth frame acquired image is determined to be the first detection box.

[0064] In one embodiment, determining the user's key points and key point location information within the first detection frame of the acquired image includes:

[0065] Based on the information from the first detection box, feature information of pose points is determined in the acquired image; wherein, the feature information includes the category information of the pose points;

[0066] Based on the category information of the attitude points, the reference point and the interaction target point are determined; thus, the key point and key point location information are obtained.

[0067] In one embodiment, the key point location information is the ordinate of the key point; then the first location information is the ordinate of the reference point, and the second location information is the ordinate of the interactive target point.

[0068] In one embodiment, determining the first motion parameter of the suspension based on the relative positional relationship between the reference point and the interactive target point includes:

[0069] In the acquired image, the reference point is determined, and the interaction target point corresponding to the user's hand is determined; wherein, the reference point includes the first reference point and the second reference point;

[0070] Based on the relative positional relationship between the interactive target point and the first reference point and the second reference point, as well as the hand posture information, the first motion parameters are determined.

[0071] Secondly, embodiments of this application provide a suspension control device, comprising:

[0072] The instruction module is used to respond to receiving an interaction instruction, acquire a captured image, and determine the key point location information of the user in the captured image; wherein, N is an integer greater than or equal to 2, and the key point location information includes first location information of a reference point and second location information of an interaction target point;

[0073] The parameter module is used to determine the first motion parameters of the suspension based on the relative positional relationship between the reference point and the interactive target point;

[0074] A motion module is used to control the suspension motion based on the first motion parameters.

[0075] In one embodiment, the instruction module is specifically used to determine key points and key point location information corresponding to the key points.

[0076] In one embodiment, the parameter module is specifically used to determine a second motion parameter of the interactive target point based on the relative positional relationship; wherein the interactive target point corresponds to the reference point of the suspension; and the second motion parameter is determined as the first motion parameter.

[0077] In one embodiment, the parameter module is specifically used to determine the relative positional relationship between the reference point and the interactive target point based on the first positional information and the second positional information; and to determine the first motion parameter of the suspension based on the relative positional relationship.

[0078] In one embodiment, the reference point includes a first reference point corresponding to the central region of the user's posture; the parameter module is specifically used to determine the first motion parameter based on the relative positional relationship between the first reference point and the interaction target point.

[0079] In one embodiment, the first motion parameter includes the motion distance of the suspension; the parameter module is specifically used to obtain the first coordinates of the reference point of the suspension; determine the first distance based on the coordinates of the first feature point and the coordinates of the first reference point, and determine the second distance based on the coordinates of the first reference point and the coordinates of the interactive target point; and determine the motion distance of the suspension according to the first coordinates and the first ratio between the second distance and the first distance.

[0080] In one embodiment, the parameter module is specifically used to determine the first distance based on the ordinate of the first feature point and the ordinate of the first reference point; and to determine the second distance based on the ordinate of the first reference point and the ordinate of the interactive target point.

[0081] In one embodiment, the first motion parameter includes the motion distance of the suspension; then the parameter module is specifically used to obtain the first coordinates of the reference point of the suspension; determine the first distance based on the coordinates of the first feature point and the coordinates of the first reference point, and determine the second distance based on the coordinates of the first reference point and the coordinates of the interactive target point; and determine the motion distance of the suspension according to the first coordinates and the first ratio between the second distance and the first distance.

[0082] In one embodiment, the first motion parameter includes the motion direction type of the suspension; then the parameter module is further configured to determine the motion direction type as a first direction type in response to a first difference between the ordinate of the second position information and the ordinate of the first reference point being less than a preset first target threshold; or, in response to the first difference being greater than or equal to the first target threshold, determine the motion direction type as a second direction type; wherein the first direction corresponding to the first direction type is opposite to the second direction corresponding to the second direction type.

[0083] In one embodiment, the parameter module is further configured to determine a fourth distance corresponding to the motion direction type and an extreme height of the suspension corresponding to the motion direction type; determine a second distance; wherein the second distance is the distance between the coordinates of the first reference point and the interactive target point; determine the ratio of the second distance to the fourth distance as a second ratio; wherein the second ratio corresponds to the motion direction type; and determine the motion distance based on the second ratio, a preset suspension control coefficient, and the extreme height corresponding to the motion direction type.

[0084] In one embodiment, the parameter module is further configured to determine the distance between the first feature point and the first reference point as the fourth distance in response to the motion direction type being a first direction type; or, in response to the motion direction type being a second direction type, determine the distance between the first reference point and the second feature point as the fourth distance; wherein the second feature point corresponds to the bottom region of the user posture, and the second feature point is obtained by processing the second reference point and the first reference point through a second preset rule.

[0085] In one embodiment, the parameter module is further configured to: Among them, D gh is the distance of motion corresponding to the type of motion direction. max The extreme height corresponding to the motion direction type is given by k, which is a preset suspension control coefficient, D4 is the fourth distance corresponding to the motion direction type, and D2 is the second distance.

[0086] In one embodiment, the instruction module is specifically used to determine a first detection box containing the user in the acquired image by matching a first type of detection box obtained from target detection with a second type of detection box obtained from trajectory prediction; and to determine the user's key points and key point location information in the first detection box of the acquired image.

[0087] In one embodiment, the instruction module is specifically used to determine the first detection box of the user in the nth frame of the captured image based on the first type detection box in the (n-1)th frame of the captured image, in response to the matching parameter between the first type detection box and the second type detection box in the nth frame of the captured image being less than a preset first matching parameter threshold; where n is an integer and 2≤n≤N.

[0088] In one embodiment, the instruction module is specifically used to perform target detection on the acquired image to obtain a first type of detection box; wherein, the first type of detection box includes a second detection box in the (n-1)th frame of the acquired image and a third detection box in the nth frame of the acquired image, where n is an integer and 2≤n≤N; predicting the trajectory of the user in the second detection box in the (n-1)th frame of the acquired image to obtain a fourth detection box in the nth frame of the acquired image; for the nth frame of the acquired image, in response to the matching parameter between the third detection box and the fourth detection box being less than a preset second matching parameter threshold, correcting the third detection box in the nth frame of the acquired image based on the second detection box to obtain the first detection box.

[0089] In one embodiment, the instruction module is specifically configured to determine a first number of captured images prior to the nth frame captured image as a target image sequence based on the time stamp carried by the captured image; in response to the captured images in the target image sequence including the first detection box, the third detection box in the nth frame captured image is corrected to obtain the first detection box.

[0090] In one embodiment, the instruction module is further configured to determine the third detection box of the nth frame image as the first detection box in response to the matching parameter between the third detection box and the fourth detection box being greater than or equal to the second matching parameter threshold.

[0091] In one embodiment, the instruction module is further configured to determine feature information of pose points in the acquired image based on the information of the first detection box; wherein the feature information includes the category information of the pose points; and determine the reference point and the interaction target point according to the category information of the pose points; thus obtaining the key point and key point location information.

[0092] In one embodiment, the parameter module is further configured to determine the reference point and the interaction target point corresponding to the user's hand in the acquired image; and to determine the first motion parameter based on the relative positional relationship between the interaction target point and the reference point, and the posture information of the hand.

[0093] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect and any embodiment.

[0094] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the methods described in the first aspect and any embodiment.

[0095] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods described in the first aspect and any embodiment.

[0096] The aforementioned suspension control method, device, electronic device, and storage medium, by acquiring key point position information of the user in an image, determine a first motion parameter for the suspension that matches the user's height. Based on this first motion parameter, the suspension's movement is controlled, enabling the suspension to more accurately understand the user's interaction intentions and adaptively move with changes in the user's posture. This method improves the interaction effect and user experience while also effectively enhancing the suspension's response stability, avoiding problems caused by insufficient suspension response stability or mismatch with the user's interaction intentions due to users being too tall or too short (e.g., children).

[0097] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The purposes and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit this disclosure. Attached Figure Description

[0098] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0099] Figure 1 This is a flowchart illustrating a suspension control method in one embodiment;

[0100] Figure 2A This is a schematic diagram of a user pose containing a first feature point in one embodiment;

[0101] Figure 2B This is a schematic diagram showing the position of the first reference point in one embodiment;

[0102] Figure 3 This is a schematic diagram of a user pose containing a second feature point in one embodiment;

[0103] Figure 4 This is a schematic diagram of the user's interaction target point in one embodiment;

[0104] Figure 5 This is a flowchart illustrating the process of determining the first detection box containing the user in multiple frames of acquired images in one embodiment.

[0105] Figure 6 This is a schematic diagram of a suspension control method in one embodiment;

[0106] Figure 7 This is a structural block diagram of a suspension control device in one embodiment;

[0107] Figure 8 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0108] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0109] It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of this application. Therefore, the drawings only show components relevant to this application and are not drawn according to the actual number, shape, and size of components in implementation. In actual implementation, the form, quantity, and proportion of each component can be arbitrarily changed, and the component layout may also be more complex. The structures, proportions, sizes, etc., shown in the accompanying drawings are only used to complement the content disclosed in the specification for those skilled in the art to understand and read, and are not intended to limit the implementation conditions of this application. Therefore, they have no substantial technical significance. Any modification to the structure, change in the proportional relationship, or adjustment of the size, without affecting the effect and purpose that this application can produce, should still fall within the scope of the technical content disclosed in this application. At the same time, the terms such as "upper," "lower," "left," "right," "middle," and "one" used in this specification are only for clarity of description and are not intended to limit the scope of implementation of this application. Changes or adjustments in their relative relationships, without substantially changing the technical content, should also be considered within the scope of implementation of this application.

[0110] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the document does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0111] As illustrated herein, unless the context clearly indicates otherwise, the words “a,” “an,” “an,” and / or “the” do not specifically refer to the singular and may also include the plural. Generally speaking, the terms “comprising” and “including” only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0112] The definitions used herein, such as the terms “having,” “may have,” “comprising,” or “may include,” indicate the presence of the corresponding function, operation, element, etc., and do not limit the presence of one or more other functions, operations, elements, etc. Furthermore, it should be understood that the terms “comprising” or “having” as used herein indicate the presence of the features, figures, steps, operations, elements, components, or combinations thereof described in the specification, without excluding the presence or addition of one or more other features, figures, steps, operations, elements, components, or combinations thereof.

[0113] The prefixes such as "first" and "second" used in this application embodiment are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. For statements regarding the described objects, please refer to the claims or the context of the embodiments. The use of such prefixes should not constitute unnecessary limitations. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0114] To facilitate understanding of the technical solutions provided in the embodiments of this application, the design concept of the embodiments of this application will be introduced first below:

[0115] When a user interacts with a vehicle, the stability of the suspension response (e.g., active suspension) is crucial. Especially when the user is outside the vehicle, the information collected by the vehicle's onboard data acquisition devices and how this information is processed significantly impacts the errors in the suspension control parameters and the stability of the suspension response. Taking image acquisition devices as an example, in processing the acquired images to identify the user's interaction intent and control the suspension, the user's height, as well as the distance and angle between the user and the onboard image acquisition devices, become major factors affecting the suspension control parameters. If the suspension directly responds to users of different heights, the excessive movement of the suspension's baseline can lead to poor interaction, insufficient accuracy in user control, and consequently, poor suspension response stability. The aforementioned suspension baseline refers to the horizontal line where the suspension's reference point is located.

[0116] Therefore, this application provides a suspension control method that determines the first motion parameters of the suspension by determining the user's key point position information and using the relative positional relationship between the interactive target point and the reference point among the key points. This avoids the problem of poor suspension response stability caused by large differences in the generation of the first motion parameters of the suspension when the user is tall or short.

[0117] In one embodiment, such as Figure 1 As shown, a suspension control method is provided, which includes the following steps:

[0118] Step 101: In response to receiving the interaction command, acquire the captured image and determine the key point location information of the user in the captured image.

[0119] The key point location information includes first location information of the reference point and second location information of the interactive target point.

[0120] Specifically, the representation of the above-mentioned key point location information includes, but is not limited to, at least one of two-dimensional coordinates, three-dimensional coordinates, polar coordinates, and latitude and longitude coordinates.

[0121] In one embodiment, the image acquisition can be obtained by acquiring a user image using an image acquisition device after receiving an interaction command. Specifically, the interaction command may contain user information, so the user information can be read from the interaction command first, and then the relative positional relationship between the user and the vehicle can be determined based on the location information in the user information, so as to determine the image acquisition device mounted on the vehicle body, and then use the image acquisition device to acquire an image of the target area.

[0122] The user information contained in the interactive command includes at least the user's location information, which is used to determine the relative positional relationship between the user and the vehicle. From multiple image acquisition devices mounted at various locations on the vehicle body, an image acquisition device is selected for image acquisition. The target area in the interactive command corresponds to the location information; therefore, this target area can be used to determine the shooting parameters of the image acquisition device, so that the resolution of the acquired image is greater than a preset resolution threshold.

[0123] In one embodiment, the suspension's response accuracy and stability can be improved by enhancing the accuracy of the user's key point location information. Specifically, the accuracy of the key point location information can be improved by acquiring N frames of images to determine the user's key point location information within them.

[0124] Optionally, upon receiving the interactive command, the image acquisition device mounted on the vehicle body can be powered on immediately to acquire images in the directions corresponding to the lenses of the image acquisition device, thereby obtaining images of the area surrounding the vehicle. Then, images are selected from the image frames based on the position information and the pre-specified target pose.

[0125] The pre-specified target pose can be used to verify the pose of the target in the image frame, so as to quickly identify the first image frame containing the user in a multi-frame image. Then, all image frames after the first image frame can be used as acquired images.

[0126] Optionally, after recognizing the aforementioned first frame image, facial enhancement processing can be performed on the target in the first frame image that matches the pre-specified target pose to obtain a facial image with a resolution higher than a resolution threshold. By verifying this facial image, the accuracy of user recognition can be improved.

[0127] Then, the image frames following the first image are used as the acquired images.

[0128] In one embodiment, the aforementioned N frames of acquired images may carry a timestamp. This timestamp may, for example, be the acquisition time of the acquired images.

[0129] The N frames of images can be sorted according to their timestamps.

[0130] The above N frames of captured images include at least two consecutive frames.

[0131] In one embodiment, the N frames of acquired images can be consecutive image frames, that is, the acquisition time interval between image frames is equal and the acquisition time interval is less than a preset first time threshold, such as 1 second or 1 millisecond.

[0132] In one embodiment, the N captured images may include consecutive image frames and non-consecutive image frames. For example, in the N captured images, the acquisition time interval between frames a is equal and the acquisition time interval is less than a preset first time threshold, and the acquisition time interval between frames b is different and less than a preset second time threshold. Where N = a + b.

[0133] Furthermore, the key points corresponding to the aforementioned key point location information may include: an interaction target point and a reference point corresponding to the user's hand. The reference point includes a first reference point corresponding to the central region of the user's posture.

[0134] Optionally, the reference point may also include a second reference point corresponding to the shoulder area of ​​the user's posture.

[0135] In one embodiment, the key points may further include: a first feature point corresponding to the top region of the user's pose, and a second feature point corresponding to the bottom region of the user's pose.

[0136] In one embodiment, key points may include the aforementioned interactive target point, reference point, a first feature point corresponding to the top region of the user's posture, and a second feature point corresponding to the bottom region of the user's posture.

[0137] Preferably, the aforementioned user posture is a standing posture.

[0138] The central region of the aforementioned user posture can be the user's waist area when the user is standing.

[0139] The top area of ​​the user's posture can preferably be the area where the user's hand is located when the hand is raised to its highest position in a standing posture.

[0140] The bottom area is preferably the area where the user's arm hangs naturally when standing.

[0141] In one embodiment, the first feature point is obtained by processing the second reference point and the first reference point using a first preset rule.

[0142] In one embodiment, similar to the first feature point, the second feature point can be obtained by processing the aforementioned second reference point and the first reference point through a second preset rule.

[0143] The aforementioned reference point can be used to determine the magnitude of change in the position of the interactive target point, thereby determining the suspension's baseline along with the first motion parameter of the suspension along its longitudinal axis. Here, the baseline is the horizontal line where the reference point is located. For example, the aforementioned first motion parameter can be determined based on the relative positional relationship between the first reference point and the interactive target point.

[0144] Because the height change of the reference line during suspension movement is actually at the millimeter level—for example, changes of 10 mm, 20 mm, or 30 mm—to further improve the suspension's response stability and user experience, and to avoid the suspension's reference point position changing too slightly to accommodate large changes in the user's posture, thus degrading the user experience, preferably, in this embodiment, the reference point is determined by the height of a region located at a preset position in the user's standing posture.

[0145] The following explanation first assumes the user is standing, and describes the positional relationship between the reference point and the aforementioned first and / or second feature points:

[0146] In one embodiment, the aforementioned first feature point corresponds to the top region of the user's posture. Preferably, the top region of the user's posture indicates the highest area that the user's hand can reach when standing. Therefore, the top region is the area corresponding to the hand when the user is standing and the arm is in a raised position.

[0147] Figure 2A This is a schematic diagram illustrating a user gesture provided in an embodiment of this application. Please refer to... Figure 2A The first feature point is marked with "★"; then the first feature point can be the center point of the user's hand when the hand is raised to its highest position.

[0148] The first reference point is marked "●1". This first reference point corresponds to the central region of the user's posture. This central region at least partially overlaps with the waist. Therefore, the central region of the user's posture can be the area corresponding to the user's waist position when the user is standing. Please refer to... Figure 2A , Figure 2B The first reference point is marked "●1". This first reference point can be the intersection of the reference line corresponding to the user's arm when it hangs naturally and the horizontal reference line corresponding to the user's waist. The middle area is the overlapping area between the area extending along the downward direction of the user's arm and the area extending laterally (i.e., horizontally) along the user's waist. The second reference point is marked "●2", and this second reference point corresponds to the shoulder area of ​​the user's posture.

[0149] In one embodiment, the aforementioned second feature point corresponds to the area located at the bottom of the user's posture when the user is standing. Please refer to... Figure 3The second feature point is marked with "★", which can be the center point of the hand when the user is standing with their arm hanging naturally. Therefore, the bottom area represents the hand area when the user is standing with their arm hanging naturally. The center point of the hand can be the geometric center of the palm when the user's fingers are spread; it can also be the geometric center of the fist when the hand is clenched, etc.

[0150] In one embodiment, it can be determined by combining relevant detection algorithms with preset rules. The explanation is as follows:

[0151] First, the pre-trained first detection algorithm can be used to process the acquired image: based on the information of the first detection box, the feature information of the pose points in the acquired image is determined.

[0152] The feature information includes the coordinates of the pose point and the pose point's category information. The first detection algorithm could be, for example, YOLO-Pose. The category information indicates the user's body part corresponding to the pose. For example, the aforementioned category information could be at least one of the following: waist, hand, head, elbow, and shoulder.

[0153] Then, based on the aforementioned attitude point category information, reference points and interaction target points are determined; and, the first reference point and the second reference point among the reference points can be processed using corresponding preset rules to obtain the first feature point and the second feature point.

[0154] In this way, key points can be obtained. The feature information of the aforementioned attitude points can also include the coordinates of the attitude points, so the location information of the key points can be determined based on the coordinates of the reference points and the coordinates of the interaction target points in the category information.

[0155] For example, to obtain key point location information, a first reference point corresponding to the waist and a second reference point corresponding to the shoulder can be determined firstly based on category information. Then, the height difference H between the first and second reference points is processed by a first preset rule in the preset rules to obtain the first feature point.

[0156] Furthermore, the height difference H between the second reference point and the first reference point is processed by the second preset rule in the preset rules to obtain the second feature point.

[0157] To improve the efficiency and accuracy of determining key point location information, and to save terminal computing power, in one embodiment, the ordinate of the aforementioned key point can be directly determined as the key point location information. This ordinate can be obtained through coordinates in the image coordinate system and / or the physical coordinate system.

[0158] For example, the aforementioned category information includes the shoulder and waist. The reference points include at least a second reference point corresponding to the shoulder and a first reference point corresponding to the waist. First, based on the aforementioned category information, the second reference point corresponding to the shoulder and its ordinate L1 can be determined; then, the ordinate of the first feature point can be determined using corresponding preset rules. Also, based on the aforementioned category information, the ordinate L2 of the first reference point corresponding to the waist is determined, and the ordinate of the second feature point is determined using corresponding preset rules. The aforementioned method utilizes the category information of the reference points to determine the ordinate L1 of the second reference point corresponding to the shoulder and the ordinate L2 of the first reference point corresponding to the waist.

[0159] Let H be the absolute value of the difference between L1 and L2. H represents the distance between the shoulder and the waist.

[0160] The first preset rule for determining the ordinate y1 of the first feature point is: y1 = α1 × H + L2; where α1 is a preset first coefficient. This first coefficient α1 can be, for example, 1.6; that is, y1 = 1.6H + L2.

[0161] The ordinate y2 of the first reference point satisfies: y2 = L2;

[0162] The second preset rule for determining the ordinate y3 of the second feature point is: y3 = L2 - α2 × H. Here, α2 is a preset second coefficient. This second coefficient α2 can be, for example, 0.3; that is, y3 = L2 - 0.3H.

[0163] Thus, the ordinate of the first feature point can be obtained by using the second reference point corresponding to the shoulder and the first reference point corresponding to the waist, according to the first preset rule y1=1.6H+L2.

[0164] Furthermore, the ordinate of the second feature point can be obtained according to the second preset rule y3 = L2 - 0.3H.

[0165] Furthermore, to improve the response stability of the suspension, in one embodiment, the response stability of the suspension can be improved by enhancing the accuracy of the recognized user posture. Because changes in user posture, especially when the user interacts with the vehicle through posture changes, can accurately reflect the user's interaction intentions; and because the hand is one of the easiest parts of the user's limbs to identify compared to some joints, the requirements for image acquisition conditions are relatively low, especially regarding the position of the hand. Therefore, it also has the advantages of low requirements for lighting conditions in the shooting environment and low requirements for the shooting parameters of the image acquisition device, making it widely applicable. Therefore, in one possible implementation, please refer to... Figure 4The center point of the user's hand can be used as the interaction target point. This allows for the identification of the user's hand after user identification, in order to determine the interaction target point and / or secondary location information of the interaction target point.

[0166] Therefore, the reference point mentioned above is a pre-specified coordinate point, while the interactive target point changes with the user's position and / or posture.

[0167] For example, the source of the received interactive instructions may be a voice command issued by the user.

[0168] For example, the interaction command can be a preset gesture command formed by the user through a pre-specified gesture posture.

[0169] For example, interactive commands can also be sent via virtual buttons on the vehicle's infotainment screen controlled by the user.

[0170] For example, the interaction command may be sent by the user's portable terminal, which sends a corresponding interaction start identifier.

[0171] The transmission of these interactive commands may take the form of, but is not limited to, sending via Bluetooth signals and / or wireless signals.

[0172] Furthermore, the interactive command can instruct the activation of the gesture control suspension motion function, so that the vehicle, upon receiving the interactive command, begins acquiring images to determine the user's gesture information in the acquired images. This gesture information may include the relative positional relationship between the reference point and the interaction target point described in step 102.

[0173] Step 102: Determine the first motion parameters of the suspension based on the relative positional relationship between the reference point and the interactive target point.

[0174] Specifically, the relative positional relationship between the reference point and the interactive target point can be determined based on the first positional information of the reference point and the second positional information of the interactive target point.

[0175] Then, based on this relative positional relationship, the first motion parameters of the suspension are determined.

[0176] Understandably, suspension response stability refers to the suspension's ability to react quickly and accurately, maintaining vehicle posture and stability, when the user's interaction intent changes. Determining the suspension's first motion parameter based on the relative positional relationship between a reference point and the interaction target point effectively improves the accuracy of recognizing changes in user posture, thereby enhancing suspension response stability. This avoids the problem of directly selecting a pre-specified user position as the interaction target point. When comparing the relative positional relationship between this interaction target point and a pre-specified reference point (with fixed coordinates), different users have different heights. For example, taller users might intend to move downwards, but their interaction target point, after moving downwards, would remain at a higher coordinate, causing the suspension to move upwards incorrectly, resulting in reduced suspension response stability. Conversely, shorter users might intend to move upwards, but their interaction target point, after moving upwards, would remain at a lower coordinate, causing the suspension to move downwards incorrectly, significantly reducing suspension response stability.

[0177] The aforementioned first motion parameter includes, but is not limited to, the type of motion direction of the suspension in the vertical direction and / or the motion distance. The unit of this motion distance may be, for example, millimeters.

[0178] In one embodiment, the first motion parameter includes the motion distance of the suspension. This motion distance can indicate the motion distance of the reference point when the location of the reset point is used as the reference coordinate for the suspension motion. The reference line where the reference point is located moves synchronously with the reference point. Alternatively, the motion distance can also be the distance the reference point moves away from the reset point along the longitudinal axis.

[0179] The reference point mentioned above is explained as follows: In this embodiment, the reference point is located on the suspension. The reference point, the horizontal line on which the reference point is located (i.e., the reference line on which the reference point is located), and the suspension move together.

[0180] The reset point is explained as follows: In this embodiment, the reset point is a stationary point. Before the user interacts with the vehicle, the suspension reset point may coincide with the reference point. When the user begins to interact with the vehicle, or after the user begins to interact with the vehicle, the suspension moves, and the reset point does not move with the movement of the suspension.

[0181] In one embodiment, the first motion parameter includes the motion direction type of the suspension. This motion direction type indicates the direction in which the suspension's reference point moves away from the reset point.

[0182] That is, with the reset line as a reference, the direction in which the suspension moves away from the reset line in a direction perpendicular to the ground. The reset line indicates the horizontal line where the reset point is located.

[0183] Step 103: Control the suspension movement based on the first motion parameters.

[0184] Specifically, a corresponding control signal can be generated based on the first motion parameter, and the control signal can be sent to the MCU (Microcontroller Unit) in the suspension domain so that the MCU controls the suspension movement and realizes human-vehicle interaction.

[0185] Furthermore, the aforementioned baseline is a horizontal reference line where the reference point is located; in one embodiment, the baseline of the suspension is controlled to follow the reference point along the longitudinal axis direction according to the first motion parameter.

[0186] The vertical axis can be the y-axis in a Cartesian coordinate system, an image coordinate system, or a physical coordinate system. Here, the coordinate system is consistent with the coordinate system used to determine the key point location information described in step 101.

[0187] It should be noted that the suspension control method provided in this application embodiment is particularly applicable to the control of active suspension.

[0188] Since the suspension typically moves as an entire suspension moving upwards or downwards, the aforementioned first motion parameter of the suspension can be achieved by controlling the movement of the suspension's reference point based on the first motion parameter. The movement of the reference point then drives the overall movement of the suspension, thereby controlling the suspension's movement.

[0189] By controlling the suspension movement based on the relative position changes between the user's key points, adaptive control can be achieved for users of different heights. This allows for accurate understanding of the user's interaction intentions and enables interaction with the user through suspension movement control. It avoids the problem of unstable suspension response caused by the reference point coordinates being too high or too low when the suspension's reset horizontal line moves with the user's posture due to height differences.

[0190] For further information, please continue to refer to [link / reference]. Figure 2A , Figure 2B Because the suspension moves with its reference point during operation, the user can control the suspension's reference point through an interactive target point, thus controlling the suspension. This interactive target point corresponds to the suspension's reference point. Specifically, based on the aforementioned relative positional relationship, a second motion parameter for the interactive target point can be determined first. This reference point corresponds one-to-one with the interactive target point. The second motion parameter includes the motion distance and / or motion direction type. Then, based on the second motion parameter, the aforementioned first motion parameter can be determined.

[0191] For example, if the second motion parameter is the motion distance, then the motion distance of the first motion parameter can be determined based on the motion distance in the second motion parameter.

[0192] For example, if the second motion parameter is the motion direction type, then the motion direction type in the first motion parameter can be determined based on the motion direction type in the second motion parameter.

[0193] For example, if the second motion parameter includes motion distance and motion direction type, then the motion distance in the first motion parameter can be determined based on the motion distance in the second motion parameter. And the motion direction type in the first motion parameter can be determined based on the motion direction type in the second motion parameter.

[0194] Thus, by using the relative positional relationship between the aforementioned reference point and the interactive target point, the motion change of the interactive target point can be used as a second motion parameter and mapped to the reference point, thereby determining the first motion parameter of the suspension (i.e., the first motion parameter of the suspension's reference point), enabling the suspension to move in accordance with the positional change of the user's interactive target point.

[0195] Furthermore, the following embodiment illustrates the determination of the motion distance in the first motion parameter:

[0196] First, obtain the first coordinates of the suspension's reference point. These first coordinates can be two-dimensional (x, y) or three-dimensional (x, y, z). To improve the efficiency of determining the first motion parameter, two-dimensional coordinates are preferred. The following explanation uses two-dimensional coordinates as an example.

[0197] Then, based on the coordinates of the first feature point and the coordinates of the first reference point, the first distance D1 is determined; and based on the coordinates of the first reference point and the coordinates of the interactive target point, the second distance D2 is determined.

[0198] Finally, based on the first coordinate and the first ratio between the second distance and the first distance...

[0199] The suspension travel distance is determined by multiplying the first ratio, the first coordinate, and the preset suspension control coefficient.

[0200] In one embodiment, the first distance D1 and the second distance D2 can be calculated using the following formulas:

[0201]

[0202] Where x1 is the x-coordinate of the first feature point, x2 is the x-coordinate of the first reference point, and x... k y1 is the x-coordinate of the interactive target point, y2 is the y-coordinate of the first feature point, and y3 is the y-coordinate of the first reference point. k The vertical coordinate of the interactive target point.

[0203] Furthermore, since the suspension primarily exhibits vertical movement, i.e., it mainly moves along the longitudinal axis, in order to improve efficiency, in one embodiment, the movement height of the suspension's reference point based on the aforementioned reset point can be determined as the movement distance D. gThe unit for this distance is meters. The following is a detailed explanation:

[0204] First, a first distance can be determined based on the ordinates of the first feature point and the first reference point. For example, the first distance D1 can be the absolute value of the difference between the ordinate y1 of the first feature point and the ordinate y2 of the first reference point: D1 = |y1 - y2|.

[0205] Then, a second distance can be determined based on the ordinate of the first reference point and the ordinate of the interactive target point. For example, the second distance can be the ordinate of the interactive target point. k The absolute value of the difference between the ordinate y2 and the first reference point: D2 = |y k -y2|.

[0206] Then, based on the first ratio between the first coordinate (x0, y0) and the aforementioned first distance and second distance, The product of these factors determines the suspension's reference point based on the aforementioned reset point's motion height.

[0207] For example, the motion height can be expressed as a ratio of the ordinate y0 in the first coordinate system to a first value.

[0208] The aforementioned preset suspension control coefficient (k) is multiplied to obtain the result. For example, k = 0.1.

[0209] The aforementioned motion height indicates the height of the reference point after motion, i.e., the height of the suspension.

[0210] When the movement distance is equal to the movement height, in one embodiment, it can be calculated using the following formula:

[0211]

[0212] Among them, D g Let D be the distance traveled, k be a preset suspension control coefficient, y0 be the ordinate of the first coordinate, D1 be the first distance, and D2 be the second distance. Wherein, D... g The units of y0 and y0 are the same; for example, they can both be meters.

[0213] Alternatively, in one embodiment, it can be calculated using the following formula:

[0214]

[0215] Among them, D g h is the distance of motion. max Let D be the extreme height of the reference point, k be a preset suspension control coefficient, D1 be the first distance, and D2 be the second distance. g hmax The units must be consistent; for example, they can all be meters.

[0216] Alternatively, in one embodiment, the aforementioned movement distance can be calculated using the following formula:

[0217]

[0218] Among them, D g h is the distance of motion. max Let be the extreme height of the reference point, k be the preset suspension control coefficient, y0 be the ordinate of the first coordinate, D1 be the first distance, and D2 be the second distance. g h max The units of y0 and y0 are the same; for example, they can both be meters.

[0219] The extreme height h of the above benchmark point max A pre-specified height value can be specified. For example, it can be 0.05 meters.

[0220] Understandably, in the above formula, h max It is the maximum height change that the reference point can achieve when it moves away from the reset point in a direction perpendicular to the ground.

[0221] Furthermore, the first motion parameter of the suspension also includes the motion direction type to facilitate the determination of the suspension's motion direction. The determination of the motion direction type in the first motion parameter is explained below using a reference point as both the first feature point and the first reference point as an example:

[0222] Determine the y-coordinate in the second location information of the interactive target point k The difference between the ordinate y2 and the first reference point (i.e., y k -y2) is the first difference.

[0223] In response to the first difference being less than a preset first target threshold, that is, the first difference between the ordinate of the second position information and the ordinate of the first reference point being less than the first target threshold, the motion direction type is determined to be the first direction type.

[0224] Alternatively, in response to a first difference being greater than or equal to a first target threshold, the motion type is determined to be a second direction type. This first target threshold can, for example, be 0.

[0225] The first direction of motion corresponding to the first direction type is opposite to the second direction of motion corresponding to the second direction type.

[0226] Understandably, the meaning of "relative direction of movement" here is: taking the suspension's reset point, which coincides with the suspension's reference point before the vehicle receives the interactive command while it is stationary, as the reference point, the direction of movement of the suspension is opposite.

[0227] Thus, if the motion type is determined to be the first direction type, the suspension will move to the aforementioned reset point, meaning the position of the suspension after the motion is above the horizontal line where the reset point is located.

[0228] If the motion type is determined to be the second direction type, the suspension will move below the aforementioned reset point, meaning the position of the suspension after the motion is below the horizontal line where the reset point is located.

[0229] For example, if the first target threshold can be 0, then the first motion direction corresponding to the first direction type can be: the reference point of the suspension moves away from the aforementioned reset point toward the top of the vehicle, and the air spring corresponding to the suspension is stretched; while the second motion direction corresponding to the second direction type is: the reference point of the suspension moves away from the aforementioned reset point toward the wheel direction (i.e., the ground direction), and the air spring corresponding to the suspension is stretched.

[0230] To further improve the accuracy of the motion distance in the aforementioned first motion parameter, thereby enhancing the suspension's response stability, the determination of the motion distance will be explained below, taking the first motion parameter, which includes both motion distance and motion direction type, as an example:

[0231] First, based on the relative magnitude between the ordinate of the second position information of the interactive target point and the ordinate of the first reference point, the motion direction type is determined, and then the fourth distance corresponding to the motion direction type is determined.

[0232] For example, when the motion direction type is the first direction type, the distance between the first feature point and the first reference point is determined as the fourth distance.

[0233] When the motion direction type is the second direction type, the distance between the first reference point and the second feature point is determined as the fourth distance. This second feature point corresponds to the bottom region of the user's pose. Please refer to [further details]. Figure 3 .

[0234] Then, based on the ordinates of the first reference point and the interactive target point, the second distance is determined. The ratio between the second distance and the fourth distance is then determined as the second ratio value. Thus, the second ratio value corresponding to the motion direction type is obtained.

[0235] Finally, the travel distance is determined based on the second ratio, the extreme height corresponding to the type of motion direction, and the preset suspension control coefficient.

[0236] In the above embodiments, when determining the movement distance, the movement direction type in the first movement parameter is matched with the movement distance by combining the movement direction type, thereby further improving the response stability of the suspension when moving in different directions.

[0237] The movement height D is still taken as the movement distance.g For example, an embodiment is provided as follows:

[0238] In response to the motion direction type being the first direction type, the fourth distance D4 = |y1-y2| is determined.

[0239] Alternatively, in response to the motion direction type being the second direction type, determine the fourth distance D4 = |y3-y2|.

[0240] The second distance can still be reached via D2 = |y k -y2| is calculated. Thus, the motion height D g It can be calculated based on the following expression:

[0241]

[0242] Among them, D g h is the distance of motion corresponding to the type of motion direction. max Where k is the extreme height corresponding to the motion direction type, D4 ​​is the fourth distance corresponding to the motion direction type, and D2 is the second distance. This is the second ratio corresponding to the aforementioned type of motion direction.

[0243] Optionally, when the motion direction type is the first direction type, the extreme height corresponding to the motion direction type can be 0.07 meters.

[0244] When the motion direction type is the second direction type, the extreme height corresponding to the motion direction type is 0.05 meters.

[0245] Furthermore, regarding the determination of users in the acquired images, especially the determination of users in the first acquired image out of N acquired images, the following two embodiments are provided:

[0246] Example 1

[0247] First, the detection boxes and their labels in the image can be directly determined using an object detection algorithm. This labeling information may include, for example, the location of the detection box, the type of the object within the box, and the confidence level of that object type. Each detection box uniquely corresponds to one object. For example, the type of an object can be distinguished by a type identifier, such as 001, 002, etc. The confidence level of the object type represents the probability that the object detection algorithm identifies the object as belonging to the type corresponding to its output type identifier. For example, the confidence level can be expressed as a percentage; the higher the percentage value, the higher the probability that the object corresponds to its type.

[0248] Then, based on the labeling information of the detection boxes, the detection boxes whose target type corresponds to the user and whose confidence level meets the category threshold can be selected as target detection boxes.

[0249] Next, based on the preset posture information contained in the interaction command, a first similarity score can be determined between the first posture information of the target in the target detection box and the preset posture information to verify the user and avoid misidentification. If the first similarity score is greater than a preset first similarity threshold, the target in the detection box can be determined to be the user. Otherwise, the target in the detection box is determined not to be the user.

[0250] Example 2

[0251] First, the target detection box in the image can be determined by the target detection algorithm. Then, the facial region of the user in the target detection box can be enhanced to obtain a high-resolution facial image and realize high-definition restoration of the user's facial image.

[0252] Next, a pre-stored authorized user facial image is retrieved. A second similarity is determined between the pre-stored authorized user facial image and the high-resolution facial image obtained through the aforementioned enhancement process. If the second similarity is greater than a preset second similarity threshold, the target in the detection box is determined to be a user. Otherwise, the target in the detection box is determined not to be a user.

[0253] Furthermore, while identifying the user in the aforementioned acquired image (e.g., the first frame acquired image), the detection bounding box information containing the user and the user's motion information in the acquired image can also be obtained. The detection bounding box information includes at least the user's position in the image. The motion information includes at least the user's motion direction. This motion direction can be determined by using a static target in the acquired image or the image coordinate system as a reference.

[0254] In this embodiment, suspension control is achieved through human-vehicle interaction. To further improve the accuracy of the identified user and user key point location information, thereby enhancing the suspension's response stability, in one embodiment, in the same frame of acquired images, detection boxes containing users obtained by different algorithms are matched to achieve verification purposes. This avoids the problem of false detections being difficult to detect due to some acquired images having incorrect user identification while others have correct user identification in N frames of acquired images.

[0255] Specifically, firstly, in the acquired image, a first detection box containing the user can be determined by matching the first type of detection box obtained from target detection with the second type of detection box obtained from trajectory prediction. Thus, each acquired image contains a first type of detection box and a second type of detection box corresponding to the user, obtained using different detection algorithms. If the first type of detection box successfully matches the second type of detection box, then the first type of detection box can be determined as containing the user.

[0256] Then, within the first detection box in the acquired image, the user's key points and key point location information are determined. Specifically, when determining the user's key points, the acquired image containing the first detection box, or the acquired image containing the first detection box and the detection box information of the first detection box, can be input into a pre-trained key point detection algorithm to obtain the user's key points and key point location information.

[0257] The pre-trained keypoint detection algorithm may include, but is not limited to, CPM (Convolutional Pose Machines) and / or YOLO-Pose (You Only Look Once Pose, YOLO human pose estimation algorithm).

[0258] In this embodiment, for the same frame of captured image, different types of detection boxes obtained by two types of algorithms are matched to achieve prediction and to verify the predicted and detected targets, thereby improving the accuracy of tracking users during user-vehicle interaction. Especially in open scenarios where the user is outside the vehicle, interfering targets (e.g., pedestrians) are likely to appear near the user. This method in this embodiment can significantly improve the accuracy of tracking users by avoiding user misidentification, thereby improving suspension response stability and effectively enhancing the user experience.

[0259] Optionally, when determining the first detection box containing the user in the acquired image, if the first type of detection box fails to match the second type of detection box, it can be assumed that the user left at the time corresponding to the acquired image, interrupting the interaction and ending the suspension control.

[0260] Preferably, to avoid errors in the generation of the first or second type of detection box; or, if the target is lost during target tracking, resulting in the failure to generate the first and / or second type of detection boxes, causing the first and second type of detection boxes to fail to match, in one embodiment, in response to the matching parameter between the first and second type of detection boxes in the nth frame of the acquired image being less than a preset first matching parameter threshold, the first detection box of the user in the nth frame of the acquired image is determined based on the first type of detection box in the (n-1)th frame of the acquired image.

[0261] The matching parameters can be obtained by matching the first type of detection boxes and the second type of detection boxes based on IOU (Intersection Over Union) using the Hungarian algorithm. Therefore, the matching parameters have only two values: one indicating a successful match and the other indicating a failed match.

[0262] Among them, the matching parameter corresponding to a successful match is greater than the first matching parameter threshold, and the matching parameter corresponding to a failed match is less than the first matching parameter threshold.

[0263] For example, if the aforementioned matching parameter is less than the first matching parameter threshold, the first detection box can be determined as any confidence level or any target category at the position corresponding to the position information of the first type detection box in the (n-1)th frame of the acquired image, based on the position information of the first type detection box in the (n-1)th frame of the acquired image.

[0264] For example, if the aforementioned matching parameter is less than the first matching parameter threshold, then based on the location information of the first type detection box containing the user in the (n-1)th frame captured image, it is first determined whether the corresponding position in the nth frame captured image contains an unmatched first type detection box. If not, then the detection box information of the first detection box in the (n-1)th frame captured image is directly used as the detection box information of the first detection box in the nth frame captured image.

[0265] If so, determine whether the number of unmatched detection boxes at the corresponding position in the n-frame acquired image is 1. If so, determine that the unmatched first type detection box is the first detection box in the n-frame acquired image.

[0266] If not, the first type of detection box that failed to match meets one of the following conditions: the target type in the detection box information does not match the user; or the target type in the detection box information matches the user, but the confidence of the target type corresponding to the user in the detection box information is lower than the threshold. In this case, the first detection box containing the user can be selected from the first type of detection boxes that failed to match according to the following rules:

[0267] Rule 1: If an unmatched first-type detection box includes a first-type detection box that matches the target type and the user, then the first-type detection box with the highest confidence among the first-type detection boxes that match the target type and the user is determined as the first detection box.

[0268] Rule 2: If the target type in the detection box information of the unmatched first type of detection box does not match the user, it can be determined that the target loss is caused by target misidentification, and the first type of detection box with the lowest confidence can be determined as the first detection box.

[0269] Furthermore, the following section provides a detailed explanation of how to determine the first detection box containing the user; please refer to [link / reference]. Figure 5 :

[0270] Step 501: Perform target detection on the acquired image to obtain the first type of detection box.

[0271] The first type of detection box includes: the second detection box in the (n-1)th frame of the acquired image and the third detection box in the nth frame of the acquired image. n is an integer, and 2≤n≤N.

[0272] Specifically, the number of first-type detection boxes in each acquired image can be 1 or more. Furthermore, the number of first-type detection boxes in each acquired image can be equal or unequal.

[0273] In step 501, while obtaining the first type of detection box, detection box information for the first type of detection box can also be generated simultaneously. This detection box information includes at least one of the target category, confidence level, and location information of the target in the first type of detection box.

[0274] The first type of detection box corresponds one-to-one with the detection box information.

[0275] In one embodiment, the first type of detection box can be obtained by performing target detection on (each) acquired image using a pre-trained target detection algorithm.

[0276] In one embodiment, the object detection algorithm includes, but is not limited to, at least one of R-CNN (Regions with Convolutional Neural Networks), Mask R-CNN (Mask Region-based Convolutional Neural Networks), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), FSSD (Feature-fused Single Shot Multibox Detector), and Efficient Det (Efficient Object Detection).

[0277] Thus, by using the aforementioned target detection algorithm, the (n-1)th frame of the acquired image is detected, and the first type of detection box that conforms to the first detection rule can be marked as the second detection box.

[0278] Similarly, by using the aforementioned target detection algorithm, the first type of detection box that conforms to the first detection rule can be marked as the third detection box.

[0279] The first detection rule includes at least the following: in the detection box information of the first detection box, the target category matches the user, and the confidence level is greater than the corresponding confidence threshold.

[0280] Step 502: Predict the user's trajectory in the (n-1)th frame of the acquired image to obtain the third information of the fourth detection box in the nth frame of the acquired image.

[0281] Specifically, the trajectory of the user in the second detection box in the (n-1)th frame of the acquired image can be predicted based on the motion information of the user in the second detection box, so as to obtain the third information of the fourth detection box in the nth frame of the acquired image.

[0282] Motion information includes the user's direction of movement and / or speed.

[0283] When predicting the trajectory, a preset trajectory prediction algorithm can be used. This trajectory prediction algorithm includes, but is not limited to, at least one of the following: Kalman filter algorithm, particle filter algorithm, and LSTM (Long Short Term Memory) network.

[0284] Thus, the fourth detection box is the predicted one, and the user's second type of detection box may be contained in the nth frame of the captured image.

[0285] The aforementioned third detection box is obtained from target detection, and may contain the user's first type of detection box in the nth frame of the acquired image.

[0286] Therefore, for each subsequent frame of the acquired image, the user's detection box information in that frame can be predicted based on the user's motion information within the first type of detection boxes in the previous m frames. The aforementioned detection box information includes at least the position information of the detection box.

[0287] Where 1≤m≤n, and m is an integer.

[0288] Step 503: For the nth frame of the acquired image, in response to the matching parameter between the third and fourth detection boxes being less than the preset second matching parameter threshold, the third detection box in the nth frame of the acquired image is corrected based on the second detection box to obtain the first detection box.

[0289] Specifically, the matching parameter indicates the degree of matching between the third and fourth detection boxes.

[0290] The matching parameters mentioned above include, but are not limited to, at least one of IOU, Euclidean distance, cosine distance, and Mahalanobis distance.

[0291] In one embodiment, the above matching parameters can be obtained by matching the third and fourth detection boxes using a pre-specified matching algorithm such as the Hungarian algorithm combined with IOU.

[0292] If the matching parameter is less than the preset second matching parameter threshold, it indicates that the third and fourth detection boxes have failed to match. Therefore, if the matching parameter is less than the preset second matching parameter threshold, it indicates that the target is lost, occluded, or the user is still in the image captured in frame n, but has been misidentified as another target.

[0293] Therefore, the detection results of the nth frame image can be corrected. Specifically, the information of the third detection box can be corrected based on the information of the second detection box, and a first detection box can be generated as the user's detection box in the nth frame image.

[0294] In one embodiment, the detection box at the same position in the image acquired in the (n-1)th frame can be directly determined as the first detection box based on the information of the second detection box in the image acquired in the nth frame.

[0295] To further improve the accuracy of user identification and thus enhance the response stability of the suspension, in one embodiment, during step 502: before determining the first detection box of the user in the nth frame of the acquired image, the first type detection box and the second type detection box in the (n-1)th frame of the acquired image can be verified firstly. That is, through steps 501-503, the first type detection box containing the user is determined in the (n-1)th frame of the acquired image, and the verification of the second detection box is realized, thereby obtaining the first detection box in the (n-1)th frame of the acquired image.

[0296] Then, trajectory prediction is performed based on the motion information of the target in the first detection box in the (n-1)th frame of the acquired image to obtain the fourth detection box in the nth frame of the acquired image.

[0297] Similarly, the trajectory prediction for each frame of the captured image is based on the motion information of the user in the previous captured image containing the user (i.e., the motion information of the target in the first detection box), thereby improving the accuracy of trajectory prediction in each captured image by improving the accuracy of the fourth detection box.

[0298] To further improve the accuracy of user identification and thus enhance the suspension's response stability, in one embodiment, when the matching parameter is less than a preset second matching parameter threshold, false detections or target loss caused by detection jitter or jumps in the currently acquired image can be identified first. Specifically, in response to the matching parameter being less than the preset second matching parameter threshold, the third detection box can be corrected using the following method:

[0299] First, based on the time stamp carried by each frame of the N-frame acquisition images, the first number of acquisition images before the n-th frame are determined as the target image sequence.

[0300] Then, in response to the fact that all acquired images in the target image sequence include the first detection box, that is, each acquired image in the target image sequence contains a detection box, it can be determined that a false detection or loss occurred in the nth frame acquired image. Therefore, it is determined to correct the third detection box: based on the second detection box, the third detection box in the nth frame acquired image is corrected to obtain the first detection box.

[0301] Alternatively, if the number of captured images containing the first detection box in the target image sequence is less than a preset second number, then the captured image in that frame does not contain the user, and the suspension control is determined to end.

[0302] In one embodiment, after step 502, the method further includes:

[0303] In response to the aforementioned matching parameter being greater than or equal to the aforementioned second matching parameter threshold, the third detection box in the nth frame of the acquired image is determined as the first detection box.

[0304] In one embodiment, while determining the first detection box and the third detection box, the information of the first detection box and the information of the second detection box are also determined.

[0305] The information in the first detection box includes the identifier of the target within the detection box: a first identifier. The information in the third detection box includes the identifier of the target within the detection box: a third identifier. At this point, the identifiers of the targets in the first and third detection boxes are different. However, when the third detection box is determined to be the first detection box, it is actually determined that the target in the third detection box and the target in the first detection box are the same target. Thus, in one embodiment, after determining that the third detection box is the first detection box, the third identifier of the third detection box can be modified to the first identifier of the user in the first detection box.

[0306] The identifiers of the targets in each detection frame can be different. For example, the user identifier in the first detection frame is different from the target identifier in the second detection frame. Another example is that the target in the second detection frame is different from the target in the third detection frame. The target in the third detection frame is also different from the target in the fourth detection frame.

[0307] Furthermore, after determining the first detection box in the N frames of acquired images, the aforementioned key points and key point location information can be determined based on this first detection box:

[0308] First, the pre-trained first detection algorithm can be used to process the N frames of acquired images one by one: based on the information of the first detection box, the feature information of the aforementioned pose points is determined in the acquired images.

[0309] Then, based on the aforementioned attitude point category information, the reference point and interaction target point are determined, and the reference point is processed based on preset rules to obtain the first feature point and the second feature point; thus, the key point and key point location information are obtained.

[0310] To improve the efficiency and accuracy of determining the location information of key points, and to save terminal computing power, in one embodiment, the ordinate of the key points can be determined directly.

[0311] For example, the interaction target point corresponding to the hand can also be determined based on the category information of the pose point.

[0312] For example, based on the category information of the posture points, a third reference point corresponding to the elbow and a fourth reference point corresponding to the wrist are selected from the posture points. Then, the third and fourth reference points are connected, and the target length is extended along the direction from the third reference point to the fourth reference point to obtain a fifth reference point. The extended target length is the product of the distance between the third and fourth reference points and a third coefficient. This third coefficient can be, for example, 0.3. The resulting fifth reference point can then be determined as the interaction target point.

[0313] To further improve the accuracy of the recognition of the interactive target point, in one embodiment, the target distance between the fourth reference point and the fifth reference point corresponding to the wrist can be determined first based on the aforementioned category information. In response to the target distance being less than a preset fourth threshold, the aforementioned fifth reference point corresponding to the hand is determined as the interactive target point.

[0314] For example, the target distance is D o This can be obtained by calculating the square root of the sum of the squares of the coordinate differences between the fourth and fifth reference points corresponding to the wrist. For example, the coordinates of the fourth reference point are (x... t y t The coordinates of the fifth reference point are (x6, y6).

[0315]

[0316] Thus, in one embodiment, in order to determine the first motion parameters for controlling the suspension, a reference point and the interaction target point corresponding to the user's hand can be determined in the acquired image.

[0317] Then, the second and first reference points are processed using preset rules to obtain the reference points.

[0318] Finally, the first motion parameters can be determined based on the relative positional relationship between the interactive target point and the reference point, as well as the hand posture information.

[0319] Optionally, when the posture information remains the first gesture type, the motion direction type and / or motion distance in the first motion parameters can be determined directly based on the above relative positional relationship.

[0320] Optionally, the posture information may include a second gesture type and a third gesture type, which can then determine the motion direction type and / or motion distance based on the aforementioned relative positional relationship; and, based on the second gesture type or the third gesture type, determine the motion rate type in the first motion parameter.

[0321] For example, the first, second, and third gesture types mentioned above can each be independently selected from a fist gesture, a thumbs-up gesture, an "OK" gesture, etc.

[0322] For example, the motion rate type includes a first rate type and a second rate type. The suspension motion rate corresponding to the first rate type is greater than the suspension motion rate corresponding to the second rate type.

[0323] When determining the motion rate type, it can be based on the preset correspondence between gesture type and motion rate type:

[0324] The preset correspondences include the correspondence between the second gesture type and the first rate type, and the correspondence between the third gesture type and the second rate type. This allows for control of the suspension movement rate based on changes in different gesture types.

[0325] In summary, regarding the suspension control method provided in the embodiments of this application, please refer to... Figure 6 Through interactive commands, the system determines how to interact with the user to control the suspension's movement in the direction perpendicular to the ground (vertical axis). Specifically, the interactive object (user) is first identified in N frames of captured images: the identifier of the interactive object is determined. Subsequently, the interactive object is tracked in the captured frames: users with the same identifier are identified in each frame, and this identifier corresponds one-to-one with the target detection box where the user is located.

[0326] Then, the first motion parameter of the suspension can be determined based on the change in the ordinate of the user's interactive target point in each frame of the captured image: the first motion parameter of the suspension is obtained by mapping the distance between the ordinate of the interactive target point and the ordinates of the two reference points. In this way, the suspension motion is adaptively controlled by the continuous coordinate changes of the user's interactive target point, thereby effectively improving the response stability of the suspension.

[0327] Furthermore, the suspension control method provided in the embodiments of this application is further illustrated below with examples:

[0328] When a user needs to interact with the vehicle, they approach the vehicle to unlock it. Specifically, the user can activate suspension control via their mobile phone to send interaction commands to the vehicle.

[0329] Once the vehicle receives the interactive command, it determines the camera used to capture the aforementioned images, as well as the camera's capture parameters, based on the user information contained in the command (such as the user's location information). These capture parameters include, but are not limited to, the zoom level.

[0330] After acquiring images using the aforementioned camera, the system can further determine images containing the user based on the shooting mode selected by the user in the interactive instructions. For example, if the shooting mode is daytime mode, the target within the largest detection box in the acquired image will be identified as a potential user.

[0331] For example, if the shooting mode is a custom mode that includes the user-selected starting posture, then the target in the captured image whose starting posture is the user is a potential user.

[0332] The aforementioned shooting modes may also include a lighting mode, which allows vehicles to turn on their headlights to create a light show. This allows targets within the light projection area in the captured images to be identified as potential users.

[0333] Furthermore, the target detection box of the aforementioned potential user can be enlarged by 1.5 times to identify their face and crop out the facial region.

[0334] The facial region is enhanced to obtain a high-resolution face. Specifically, the cropped facial region is first encoded using the encoder on the vehicle and then uploaded to the cloud to save local computing resources. The cloud-based decoder then performs enhancement processing to achieve high-definition face reconstruction, and the high-resolution face is transmitted back to the vehicle.

[0335] By comparing the facial data of authorized users pre-stored in the vehicle with high-resolution facial data transmitted from the cloud, a match is made between the pre-stored facial data and the user who sent the interaction command. This on-vehicle comparison enhances user information security.

[0336] After identifying the user, the user's posture can be verified. If the user's posture matches the pre-specified human posture, the verification passes. The captured image containing the user can then be designated as the first captured image frame.

[0337] Next, suspension control can begin from the user's posture in the first captured image. The user's posture in every subsequent captured image is then used to control the suspension. For this purpose, user tracking can also begin from the first captured image, thus avoiding the efficiency reduction and suspension response delay issues caused by using the aforementioned user facial recognition method in every captured image.

[0338] In this way, for each frame of the captured images (frame 2, frame 3, ..., frame N), the first type of detection box obtained by the object detection algorithm and the second type of detection box obtained by the trajectory prediction algorithm are matched in the same captured image to achieve user tracking and avoid user identification errors. If a match fails in a certain captured image, user information from the first number of captured images before that frame is obtained as the user's position and pose information in the current captured image, thereby avoiding the problem of user loss caused by jitter and target jumps during object detection.

[0339] Based on this, the user's reference points and the hand used to determine the interaction target point can be further identified in the first frame and subsequent captured images. Specifically, the detection algorithm used for this identification can be the YOLO-Pose algorithm.

[0340] Next, using the identified second reference point corresponding to the shoulder and the first reference point corresponding to the waist, the first feature point and / or the second feature point can be determined. Specifically, in the vertical direction, the height between the first reference point corresponding to the waist and the second reference point corresponding to the shoulder can be determined as H. Then, the ordinate of the second reference point corresponding to the shoulder is added by 1.6H to obtain the ordinate of the first feature point. That is, the first reference point and the second reference point are processed through the first preset rule mentioned above to obtain the ordinate of the first feature point. This first feature point can be the highest point of the preset posture: the highest point that the user's hand can reach in a standing posture.

[0341] Furthermore, the ordinate of the second feature point can be determined by subtracting 0.3H from the ordinate of the first reference point corresponding to the waist. That is, the ordinate of the second feature point is obtained by processing the first and second reference points through the second preset rule mentioned above.

[0342] Further determine the interaction target point: This interaction target point can be obtained by directly locating (i.e., detecting) the user's hand. Alternatively, it can be obtained by adding the x-coordinate and y-coordinate of the wrist reference point to the reference point corresponding to the elbow and the wrist reference point, respectively, using 0.3 times the distance between the elbow reference point and the wrist reference point. This will give the coordinates of the palm center: the interaction target point.

[0343] Thus, the suspension's movement distance can be mapped based on the ratio between the vertical coordinate of the interactive target point and the vertical coordinate of the first human body key point (i.e., the second distance between the center point of the palm and the first key point corresponding to the waist) and the first distance, thereby adjusting the suspension height.

[0344] At the same time, the relative magnitude of the vertical coordinate of the center point of the user's palm and the vertical coordinate of the first reference point corresponding to the waist can be used to determine the type of movement of the suspension; that is, in the direction perpendicular to the ground, it moves upward or downward in the direction away from the reset line.

[0345] In this way, the suspension can dynamically move up and down to follow the user's interactive target point in each frame of the captured image, and respond efficiently and stably, effectively improving the user experience.

[0346] It should be understood that, although Figure 1 , Figure 5 , Figure 6The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 , Figure 5 , Figure 6 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0347] Based on the same inventive concept, such as Figure 7 As shown, this application embodiment provides a suspension control device, including: a command module 701, a parameter module 702, and a motion module 703, wherein:

[0348] The instruction module 701 is used to respond to receiving an interaction instruction, acquire a captured image, and determine the key point location information of the user in the captured image; wherein the key point location information includes first location information of a reference point and second location information of the interaction target point.

[0349] The parameter module 702 is used to determine the first motion parameters of the suspension based on the relative positional relationship between the reference point and the interactive target point.

[0350] Motion module 703 is used to control the suspension movement based on the first motion parameters.

[0351] In one embodiment, the instruction module 701 can be used to determine key points and the location information of the key points.

[0352] In one embodiment, the instruction module 701 is specifically used for:

[0353] In the acquired image, a first detection box containing the user is determined by matching the first type of detection box obtained from target detection with the second type of detection box obtained from trajectory prediction; in the first detection box of the acquired image, the user's key points and the key point location information are determined.

[0354] In one embodiment, the instruction module 701 is specifically used to: perform target detection on the acquired image to obtain a first type of detection box; wherein the first type of detection box includes a second detection box in the (n-1)th frame of the acquired image and a third detection box in the nth frame of the acquired image, where n is an integer and 2≤n≤N; predict the trajectory of the user in the second detection box in the (n-1)th frame of the acquired image to obtain a fourth detection box in the nth frame of the acquired image; for the nth frame of the acquired image, in response to the matching parameter between the third detection box and the fourth detection box being less than a preset second matching parameter threshold, correct the third detection box in the nth frame of the acquired image based on the second detection box to obtain the first detection box.

[0355] In one embodiment, the instruction module 701 is specifically used for:

[0356] Based on the time stamp carried by the acquired image, a first number of acquired images preceding the nth frame are determined as a target image sequence; in response to the acquired image in the target image sequence including the first detection box, the third detection box in the nth frame is corrected to obtain the first detection box.

[0357] In one embodiment, the instruction module 701 is further configured to:

[0358] In response to the matching parameter between the third detection box and the fourth detection box being greater than or equal to the second matching parameter threshold, the third detection box of the nth frame acquired image is determined to be the first detection box.

[0359] In one embodiment, the instruction module 701 is specifically used for:

[0360] In response to the fact that the matching parameter between the first type detection box and the second type detection box in the nth frame of the acquired image is less than a preset first matching parameter threshold, the first detection box of the user in the nth frame of the acquired image is determined based on the first type detection box in the (n-1)th frame of the acquired image.

[0361] In one embodiment, the instruction module 701 can also be used for:

[0362] Based on the information from the first detection box, feature information of pose points is determined in the acquired image; wherein, the feature information includes the category information of the pose points; based on the category information of the pose points, the reference point and the interaction target point are determined; thus, the key point and key point location information are obtained.

[0363] In one embodiment, the parameter module 702 may specifically be used for:

[0364] Based on the first location information and the second location information, the relative positional relationship between the reference point and the interactive target point is determined; based on the relative positional relationship, the first motion parameter of the suspension is determined.

[0365] In one embodiment, the parameter module 702 may specifically be used for:

[0366] The first distance is determined based on the ordinate of the first feature point and the ordinate of the first reference point; the second distance is determined based on the ordinate of the first reference point and the ordinate of the interactive target point.

[0367] In one embodiment, the parameter module 702 is specifically used for:

[0368] The travel distance of the suspension is determined by multiplying the ordinate of the first coordinate with the first ratio.

[0369] In one embodiment, the movement distance is determined according to the following formula:

[0370] Among them, D g h is the distance of motion. max The extreme height of the reference point is given by y0, k is the preset suspension control coefficient, y0 is the ordinate of the first coordinate, D1 is the first distance, and D2 is the second distance.

[0371] In one embodiment, the parameter module 702 is specifically used for:

[0372] Based on the relative positional relationship, a second motion parameter of the reference point of the suspension is determined; wherein the interactive target point corresponds to the reference point of the suspension; and the second motion parameter is determined as the first motion parameter.

[0373] In one embodiment, the reference point includes a first feature point corresponding to the top region of the user's posture and a first reference point corresponding to the middle region of the user's posture; the first motion parameter includes the motion distance of the suspension; then the parameter module 702 is specifically used for:

[0374] Obtain the first coordinates of the reference point of the suspension; determine a first distance based on the coordinates of the first feature point and the first reference point, and determine a second distance based on the coordinates of the first reference point and the coordinates of the interactive target point; determine the movement distance of the suspension according to the first coordinates and a first ratio between the second distance and the first distance.

[0375] In one embodiment, the reference point includes a first feature point indicating the highest point of the user's posture and a first reference point indicating the midpoint of the user's posture; the first motion parameter includes the motion direction type of the suspension; then the parameter module 702 is further configured to:

[0376] In response to a first difference between the ordinate of the second position information and the ordinate of the first reference point being less than a preset first target threshold, the motion direction type is determined to be a first direction type; or, in response to the first difference being greater than or equal to the first target threshold, the motion direction type is determined to be a second direction type; wherein the first direction corresponding to the first direction type is opposite to the second direction corresponding to the second direction type.

[0377] For specific limitations regarding the suspension control device, please refer to the limitations on the suspension control method above, which will not be repeated here. Each module in the aforementioned suspension control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the electronic device in hardware form, or stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0378] Based on the same inventive concept, please refer to Figure 8 This application also provides an electronic device. In one embodiment, the electronic device, as shown in the figure, may include a memory 801, a communication module 803, and one or more processors 802.

[0379] The memory 801 is used to store computer programs executed by the processor 802. The memory 801 mainly includes a program storage area and a data storage area. The program storage area can store the operating system, and the data storage area can store various operation instruction sets, etc.

[0380] Memory 801 may be volatile memory, such as random-access memory (RAM); memory 801 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 801 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 801 may be a combination of the above-described memories.

[0381] The processor 802 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 802 is used to implement the aforementioned suspension control method when it calls a computer program stored in the memory 801.

[0382] The communication module 803 is used to communicate with terminal equipment, site equipment or other network equipment.

[0383] This application embodiment does not limit the specific connection medium between the memory 801, communication module 803, and processor 802 described above. This application embodiment... Figure 8 The memory 801 and the processor 802 are connected via a bus 804, and the bus 804 is in Figure 8 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 804 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 8 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0384] The memory 801 stores a computer storage medium containing computer-executable instructions for implementing the method for determining suspension control according to embodiments of this application. The processor 802 executes the suspension control methods of the embodiments described above using the computer-executable instructions.

[0385] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0386] In response to receiving an interaction command, the system acquires a captured image and determines the location information of key points of the user in the captured image; wherein the location information of key points includes first location information of a reference point and second location information of an interaction target point; based on the relative positional relationship between the reference point and the interaction target point, the system determines first motion parameters of the suspension; and based on the first motion parameters, the system controls the movement of the suspension.

[0387] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0388] Determine the key points and the key point location information corresponding to the key points.

[0389] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0390] Based on the first location information and the second location information, the relative positional relationship between the reference point and the interactive target point is determined; based on the relative positional relationship, the first motion parameter of the suspension is determined.

[0391] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0392] The first distance is determined based on the ordinate of the first feature point and the ordinate of the first reference point; the second distance is determined based on the ordinate of the first reference point and the ordinate of the interactive target point.

[0393] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0394] The travel distance of the suspension is determined by multiplying the ordinate of the first coordinate with the first ratio.

[0395] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0396] In response to the fact that the matching parameter between the first type detection box and the second type detection box in the nth frame of the acquired image is less than a preset first matching parameter threshold, the first detection box of the user in the nth frame of the acquired image is determined based on the first type detection box in the (n-1)th frame of the acquired image.

[0397] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0398] The reference point includes a first feature point corresponding to the top region of the user's posture and a first reference point corresponding to the middle region of the user's posture; the first motion parameter includes the motion distance of the suspension; the first coordinates of the reference point of the suspension are obtained; a first distance is determined based on the coordinates of the first feature point and the coordinates of the first reference point, and a second distance is determined based on the coordinates of the first reference point and the coordinates of the interactive target point; the motion distance of the suspension is determined according to the first coordinates and a first ratio between the second distance and the first distance.

[0399] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0400] The reference point includes a first feature point indicating the highest point of the user's posture and a first reference point indicating the midpoint of the user's posture; the first motion parameter includes the motion direction type of the suspension; in response to a first difference between the ordinate of the second position information and the ordinate of the first reference point being less than a preset first target threshold, the motion direction type is determined to be a first direction type; or, in response to the first difference being greater than or equal to the first target threshold, the motion direction type is determined to be a second direction type; wherein, the first direction corresponding to the first direction type is opposite to the second direction corresponding to the second direction type.

[0401] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0402] In the acquired image, a first detection box containing the user is determined by matching the first type of detection box obtained from target detection with the second type of detection box obtained from trajectory prediction; in the first detection box of the acquired image, the user's key points and the key point location information are determined.

[0403] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0404] Target detection is performed on the acquired image to obtain a first type of detection box; wherein, the first type of detection box includes a second detection box in the (n-1)th frame of the acquired image and a third detection box in the nth frame of the acquired image, where n is an integer and 2≤n≤N; the trajectory of the user in the second detection box in the (n-1)th frame of the acquired image is predicted to obtain a fourth detection box in the nth frame of the acquired image; for the nth frame of the acquired image, in response to the matching parameter between the third detection box and the fourth detection box being less than a preset second matching parameter threshold, the third detection box in the nth frame of the acquired image is corrected based on the second detection box to obtain the first detection box.

[0405] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0406] Based on the time stamp carried by the acquired image, a first number of acquired images preceding the nth frame are determined as a target image sequence; in response to the acquired image in the target image sequence including the first detection box, the third detection box in the nth frame is corrected to obtain the first detection box.

[0407] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0408] In response to the matching parameter between the third detection box and the fourth detection box being greater than or equal to the second matching parameter threshold, the third detection box of the nth frame acquired image is determined to be the first detection box.

[0409] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0410] Based on the information from the first detection box, feature information of pose points is determined in the acquired image; wherein, the feature information includes the category information of the pose points; based on the category information of the pose points, the reference point and the interaction target point are determined; thus, the key point and key point location information are obtained.

[0411] In one embodiment, when the computer-executable instructions are executed by the processor, the following steps are also performed:

[0412] The key point location information is the ordinate of the key point; then the first location information is the ordinate of the reference point, and the second location information is the ordinate of the interactive target point.

[0413] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0414] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0415] In response to receiving an interaction command, the system acquires a captured image and determines the location information of key points of the user in the captured image; wherein the location information of key points includes first location information of reference points and second location information of interaction target points;

[0416] Based on the relative positional relationship between the reference point and the interactive target point, the first motion parameters of the suspension are determined;

[0417] The suspension movement is controlled based on the first motion parameter.

[0418] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0419] Determine the key point location information.

[0420] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0421] Based on the first location information and the second location information, the relative positional relationship between the reference point and the interactive target point is determined; based on the relative positional relationship, the first motion parameter of the suspension is determined.

[0422] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0423] The first distance is determined based on the ordinate of the first feature point and the ordinate of the first reference point; the second distance is determined based on the ordinate of the first reference point and the ordinate of the interactive target point.

[0424] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0425] The travel distance of the suspension is determined by multiplying the ordinate of the first coordinate with the first ratio between the first distance and the second distance.

[0426] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0427] In response to the fact that the matching parameter between the first type detection box and the second type detection box in the nth frame of the acquired image is less than a preset first matching parameter threshold, the first detection box of the user in the nth frame of the acquired image is determined based on the first type detection box in the (n-1)th frame of the acquired image.

[0428] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0429] The reference point includes a first feature point corresponding to the top region of the user's posture and a first reference point corresponding to the middle region of the user's posture; the first motion parameter includes the motion distance of the suspension; the first coordinates of the reference point of the suspension are obtained; a first distance is determined based on the coordinates of the first feature point and the coordinates of the first reference point, and a second distance is determined based on the coordinates of the first reference point and the coordinates of the interactive target point; the motion distance of the suspension is determined according to the first coordinates and a first ratio between the second distance and the first distance.

[0430] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0431] The reference point includes a first feature point indicating the highest point of the user's posture and a first reference point indicating the midpoint of the user's posture; the first motion parameter includes the motion direction type of the suspension; in response to a first difference between the ordinate of the second position information and the ordinate of the first reference point being less than a preset first target threshold, the motion direction type is determined to be a first direction type; or, in response to the first difference being greater than or equal to the first target threshold, the motion direction type is determined to be a second direction type; wherein, the first direction corresponding to the first direction type is opposite to the second direction corresponding to the second direction type.

[0432] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0433] In the N frames of the acquired images, a first detection box containing the user is determined by matching the first type of detection box obtained from target detection with the second type of detection box obtained from trajectory prediction; N is an integer greater than or equal to 2, and the key points of the user and the key point location information are determined in the first detection box of the acquired images.

[0434] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0435] Target detection is performed on the acquired image to obtain a first type of detection box; wherein, the first type of detection box includes a second detection box in the (n-1)th frame of the acquired image and a third detection box in the nth frame of the acquired image, where n is an integer and 2≤n≤N; the trajectory of the user in the second detection box in the (n-1)th frame of the acquired image is predicted to obtain a fourth detection box in the nth frame of the acquired image; for the nth frame of the acquired image, in response to the matching parameter between the third detection box and the fourth detection box being less than a preset second matching parameter threshold, the third detection box in the nth frame of the acquired image is corrected based on the second detection box to obtain the first detection box.

[0436] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0437] Based on the time stamp carried by the acquired image, a first number of acquired images preceding the nth frame are determined as a target image sequence; in response to the acquired image in the target image sequence including the first detection box, the third detection box in the nth frame is corrected to obtain the first detection box.

[0438] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0439] In response to the matching parameter between the third detection box and the fourth detection box being greater than or equal to the second matching parameter threshold, the third detection box of the nth frame acquired image is determined to be the first detection box.

[0440] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0441] Based on the information from the first detection box, feature information of pose points is determined in the acquired image; wherein, the feature information includes the category information of the pose points; based on the category information of the pose points, the reference point and the interaction target point are determined; thus, the key point and key point location information are obtained.

[0442] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0443] Based on the same inventive concept, embodiments of this application also provide a computer program product, including a computer program, which, when executed by a processor, implements the suspension control method described in any of the above claims.

[0444] The program code for executing the computer program product of this application can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0445] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0446] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0447] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0448] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of user-operated steps to be executed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0449] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A suspension control method suitable for an off-board user, characterized by, Comprise: In response to receiving the interaction instruction, acquiring a collection image, and determining the key point position information of the user in the collection image; wherein the key point position information includes first position information of a reference point, and second position information of an interaction target point; the reference point includes a first reference point corresponding to the middle region of the user posture, and a second reference point corresponding to the shoulder region of the user posture; the interaction target point corresponds to the hand of the user; Determine the first motion parameter of the suspension based on the relative position relationship between the reference point and the interaction target point; Control the suspension motion based on the first motion parameter; Wherein, the first motion parameter of the suspension is determined based on the relative position relationship between the reference point and the interaction target point, including: determining the second motion parameter of the interaction target point based on the relative position relationship; wherein the interaction target point corresponds to the reference point of the suspension, and the second motion parameter includes motion distance and / or motion direction type; the first motion parameter is determined based on the second motion parameter.

2. The method of claim 1, wherein, The first motion parameter of the suspension is determined based on the relative position relationship between the reference point and the interaction target point, including: Determine the first motion parameter based on the relative position relationship between the first reference point and the interaction target point.

3. The method of claim 2, wherein, The first motion parameter includes the motion distance of the suspension; The first motion parameter of the suspension is determined based on the relative position relationship between the reference point and the interaction target point, including: Obtain the first coordinates of the reference point of the suspension; Determine the first distance based on the coordinates of the first feature point and the coordinates of the first reference point, and determine the second distance based on the coordinates of the first reference point and the coordinates of the interaction target point; wherein the first feature point corresponds to the top region of the user posture; According to the first coordinates, and the first ratio between the second distance and the first distance, determine the motion distance of the suspension.

4. The method of claim 3, wherein, The first feature point is obtained by processing the first reference point and the second reference point through a first preset rule.

5. The method of claim 3, wherein, Determine the first distance based on the coordinates of the first feature point and the coordinates of the first reference point, and determine the second distance based on the coordinates of the first reference point and the coordinates of the interaction target point, including: Determine the first distance based on the longitudinal coordinates of the first feature point and the longitudinal coordinates of the first reference point; Determine the second distance based on the longitudinal coordinates of the first reference point and the longitudinal coordinates of the interaction target point.

6. The method of claim 5, wherein, According to the first coordinates, and the first ratio between the second distance and the first distance, determine the motion distance of the suspension, including: According to the product of the longitudinal coordinates of the first coordinates, the preset suspension control coefficient and the first ratio, determine the motion distance of the suspension.

7. The method of claim 2, wherein, The first motion parameter includes the motion direction type of the suspension; The first motion parameter of the suspension is determined based on the relative position relationship between the reference point and the interaction target point, including: In response to a first difference between a vertical coordinate in the second position information and a vertical coordinate of the first reference point being less than a preset first target threshold, determining that the motion direction type is a first direction type; or, In response to the first difference being greater than or equal to the first target threshold, determining that the motion direction type is a second direction type; The first direction type corresponds to a first direction, and the second direction type corresponds to a second direction.

8. The method of claim 7, wherein, The motion distance in the first motion parameter is determined by the following method: determining a fourth distance corresponding to the motion direction type, and an extreme height of the suspension corresponding to the motion direction type; determining a second distance; wherein the second distance is the distance between the first reference point and the interaction target point; determining a second ratio of the second distance and the fourth distance; wherein the second ratio corresponds to the motion direction type; determining the motion distance based on the second ratio, and the extreme height corresponding to the motion direction type.

9. The method of claim 8, wherein, Determining the fourth distance corresponding to the motion direction type includes: in response to the motion direction type being the first direction type, determining that the distance between the first feature point and the first reference point is the fourth distance; or, in response to the motion direction type being the second direction type, determining that the distance between the first reference point and a second feature point is the fourth distance; wherein the second feature point corresponds to a bottom region of the user posture, the second feature point is obtained by processing the second reference point and the first reference point according to a second preset rule.

10. The method of claim 9, wherein, The first motion parameter of the suspension is determined based on the relative position relationship between the reference point and the interaction target point, including: in the collected image, determining the reference point and the interaction target point; based on the relative position relationship between the interaction target point and the first reference point, the second reference point, and the posture information of the hand, determining the first motion parameter.

11. The method of claim 1, wherein, The determination of the key point position information of the user in the collected image includes: in N frames of the collected image, determining a first detection frame containing the user by matching a first type detection frame obtained by target detection and a second type detection frame obtained by trajectory prediction; wherein N is an integer greater than or equal to 2; in the first detection frame of the collected image, determining the key point of the user and the key point position information.

12. The method of claim 11, wherein, The determination of the first detection frame containing the user in the collected image by matching the first type detection frame obtained by target detection and the second type detection frame obtained by trajectory prediction includes: in response to a matching parameter between the first type detection frame and the second type detection frame in the nth frame of the collected image being less than a preset first matching parameter threshold, determining the first detection frame of the user in the nth frame of the collected image based on the first type detection frame in the (n-1) th frame of the collected image; wherein n is an integer, and 2≤n≤N.

13. The method of claim 11, wherein, The first detection frame containing the user is determined by matching a first type detection frame obtained by target detection and a second type detection frame obtained by trajectory prediction in the collected image, comprising: Target detection is performed on the collected image to obtain the first type detection frame; wherein the first type detection frame includes a second detection frame in the (n-1)th collected image and a third detection frame in the nth collected image, n is an integer, and 2≤n≤N; The trajectory of the user in the second detection frame in the (n-1)th collected image is predicted to obtain a fourth detection frame of the user in the nth collected image; For the nth collected image, in response to the matching parameter between the third detection frame and the fourth detection frame being less than a preset second matching parameter threshold, the third detection frame in the nth collected image is corrected based on the second detection frame to obtain the first detection frame.

14. The method of claim 13, wherein, The third detection frame in the nth collected image is corrected based on the second detection frame to obtain the first detection frame, comprising: Based on the time mark carried by the collected image, a first number of collected images before the nth collected image are determined as a target image sequence; In response to the collected image in the target image sequence including the first detection frame, the third detection frame in the nth collected image is corrected to obtain the first detection frame.

15. The method of claim 13, wherein, After predicting the trajectory of the user in the second detection frame in the (n-1)th collected image to obtain the fourth detection frame in the nth collected image, it further comprises: In response to the matching parameter between the third detection frame and the fourth detection frame being greater than or equal to the second matching parameter threshold, the third detection frame of the nth collected image is determined as the first detection frame.

16. The method of claim 11, wherein, In the first detection frame of the collected image, the key point of the user and the key point position information are determined, comprising: Based on the information of the first detection frame, the feature information of the posture point is determined in the collected image; wherein the feature information includes the category information of the posture point; According to the category information of the posture point, the reference point and the interaction target point are determined; then the key point and the key point position information are obtained.

17. A suspension control device for an off-board user, characterized by, Comprising: An instruction module is configured to, in response to receiving an interaction instruction, acquire a collected image and determine key point position information of a user in the collected image; wherein the key point position information includes first position information of a reference point and second position information of an interaction target point; the reference point includes a first reference point corresponding to a middle region of a user posture and a second reference point corresponding to a shoulder region of the user posture; the interaction target point corresponds to a hand of the user; A parameter module is configured to determine a first motion parameter of the suspension based on a relative position relationship between the reference point and the interaction target point; A motion module is configured to control the suspension to move based on the first motion parameter. The parameter module is specifically configured to determine a second motion parameter of the interaction target point based on the relative position relationship, wherein the interaction target point corresponds to a reference point of the suspension, and the second motion parameter comprises a motion distance and / or a motion direction type; and determine the first motion parameter based on the second motion parameter.

18. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 16 when executing the computer program.

19. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 16.

Images