Remote control parking method and system, computer equipment and storage medium

Abstract

The invention provides a remote control parking method and system, computer equipment and a storage medium, and is applied to the field of intelligent driving. The method comprises the steps of obtaining a surrounding environment detection result of a to-be-controlled vehicle according to an environment sensor; under the condition that the ambient environment detection result meets a preset condition, acquiring an ambient image of the to-be-controlled vehicle according to an image acquisition device; performing face detection according to the surrounding image to obtain a face detection result of the surrounding image; matching the face detection result with a preset face database, and under the condition that face matching succeeds, performing gesture recognition on the to-be-controlled vehicle target which succeeds in face matching to obtain a gesture recognition result; and controlling the to-be-controlled vehicle to park according to the gesture recognition result of the to-be-controlled vehicle target. Gestures of the surrounding environment of the vehicle and the target to be controlled are collected through the sensor on the vehicle to be controlled, remote control parking of the vehicle to be controlled is achieved by combining technologies such as face recognition and gesture recognition, and parking can be completed without carrying any mobile terminal equipment.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving, and in particular to a remote control parking method, system, computer device and storage medium. Background Art

[0002] Intelligent driving is a technology that allows machines to help people drive and completely replace human driving in special circumstances. Among them, one of the most popular intelligent driving technologies for unskilled drivers is automatic parking technology, which can assist drivers in automatic parking. For example, when the parking space is very narrow, there is a situation where the driver cannot complete the parking, or after parking is completed, the driver cannot smoothly open the door and get out of the car. In this case, the driver can get out of the car before the vehicle enters the parking space, and then complete the parking through automatic parking technology. Similarly, the vehicle can also be driven out of a narrow parking space through the automatic parking function. Summary of the invention

[0003] The present application provides a remote control parking method, system, computer device and storage medium to solve the technical problem of low business processing efficiency existing in the prior art.

[0004] In a first aspect, the present application provides a remote control parking method, which is applied to a vehicle to be controlled, wherein the vehicle to be controlled includes an environmental sensor and an image acquisition device, and the method includes:

[0005] Acquire the surrounding environment detection result of the vehicle to be controlled according to the environmental sensor;

[0006] When the surrounding environment detection result meets the preset conditions, the surrounding image of the vehicle to be controlled is acquired according to the image acquisition device;

[0007] Perform face detection based on surrounding images to obtain face detection results of surrounding images;

[0008] The face detection result is matched with a preset face database. If the face match is successful, gesture recognition is performed on the target vehicle to be controlled whose face match is successful to obtain a gesture recognition result.

[0009] The vehicle to be controlled is controlled to park according to the gesture recognition result of the vehicle to be controlled.

[0010] In one embodiment, the environmental sensor includes an ultrasonic radar; obtaining the detection result of the surrounding environment of the vehicle to be controlled according to the environmental sensor includes:

[0011] Control the ultrasonic radar to start, and control other control modules of the vehicle to be controlled to be in a power-off state;

[0012] Obtain surrounding environment data collected by ultrasonic radar;

[0013] According to the surrounding environment data, the surrounding environment detection result of the vehicle to be controlled is determined.

[0014] In one embodiment, the surrounding environment data includes each transmission signal of the ultrasonic radar and the echo duration of the corresponding echo signal; and determining the surrounding environment detection result of the vehicle to be controlled according to the surrounding environment data includes:

[0015] Determine the distance of the obstacle corresponding to each echo signal according to the echo duration of each transmitted signal and the corresponding echo signal;

[0016] The distances of obstacles corresponding to all echo signals are linked and analyzed to determine the surrounding environment detection results around the vehicle to be controlled.

[0017] In one embodiment, the surrounding environment detection result includes whether an obstacle is detected, the type of obstacle, the location coordinates of the obstacle, and the confidence level of the detection result; the surrounding environment detection result meets the preset conditions including:

[0018] The surrounding environment detection result includes that an obstacle is detected and the type of the obstacle is a preset obstacle type; the position coordinates of the obstacle in the surrounding environment detection result are changed and the distance between the obstacle and the vehicle to be controlled is less than the preset distance.

[0019] In one embodiment, according to the gesture recognition result of the target vehicle to be controlled, controlling the vehicle to be controlled to park includes:

[0020] Analyze the gesture recognition result of the target vehicle to be controlled, and determine the command type corresponding to the gesture of the target vehicle to be controlled;

[0021] Acquire a parking function signal of a vehicle to be controlled;

[0022] The vehicle to be controlled is controlled to park according to the command type and the parking function signal.

[0023] In one embodiment, controlling the vehicle to be controlled to park according to the command type and the parking function signal includes:

[0024] Get the command types corresponding to all gestures of the target vehicle to be controlled within a preset time;

[0025] When the command types corresponding to all gestures are preset command types and the gesture parking function signal indicates that the parking function of the to-be-controlled vehicle is normal, the to-be-controlled vehicle is controlled to park according to the control instructions corresponding to the preset command types.

[0026] In one embodiment, performing face detection based on surrounding images to obtain face detection results of surrounding images includes:

[0027] Determining at least one image to be detected from surrounding images according to a preset sampling period;

[0028] Preprocessing at least one image to be detected to remove noise points in at least one image to be detected;

[0029] Perform facial feature recognition on the at least one preprocessed image to be detected to determine a face detection result of the at least one image to be detected.

[0030] In one embodiment, gesture recognition is performed on the target vehicle to be controlled whose face is successfully matched, and a gesture recognition result is obtained, including:

[0031] Acquire a gesture image of a vehicle to be controlled by an image acquisition device;

[0032] Perform gesture detection and tracking on the gesture image to determine the gesture trajectory of the target vehicle to be controlled;

[0033] According to the gesture trajectory, the gesture recognition result of the target vehicle to be controlled is determined.

[0034] In a second aspect, the present application further provides a remote control parking system, the system is applied to a vehicle to be controlled, the vehicle to be controlled includes an environmental sensor and an image acquisition device; the system includes a cabin-parking integrated subsystem and a vehicle control subsystem;

[0035] The cabin-parking integrated subsystem is used to control the environmental sensor and the image acquisition device, and determine the gesture recognition result of the target vehicle to be controlled corresponding to the vehicle to be controlled based on the data collected by the environmental sensor and the image acquisition device, and send the gesture recognition result to the vehicle control subsystem;

[0036] The vehicle control subsystem is used to control the operation of the vehicle to be controlled according to the received gesture recognition results.

[0037] In one embodiment, the cabin-mooring integrated subsystem includes an environment detection module, a face recognition module and a gesture recognition module; wherein,

[0038] An environment detection module is used to obtain the surrounding environment detection results of the vehicle to be controlled according to the environment sensor;

[0039] The face recognition module is used to obtain the surrounding images of the vehicle to be controlled according to the image acquisition device when the surrounding environment detection results meet the preset conditions; perform face detection according to the surrounding images to obtain the face detection results of the surrounding images; and match the face detection results with the preset face database;

[0040] The gesture recognition module is used to perform gesture recognition on the target vehicle to be controlled whose face is successfully matched, obtain the gesture recognition result, and send the gesture recognition result to the vehicle control subsystem when the face matching is successful.

[0041] In one of the embodiments, the cabin-parking integrated subsystem is also used to obtain a status signal from the vehicle control subsystem and determine whether the status signal is normal. If the status signal is abnormal, an exit control instruction is sent to the vehicle control subsystem.

[0042] In a third aspect, the present application further provides a computer device, the computer comprising: a processor, and a memory communicatively connected to the processor;

[0043] Memory stores computer-executable instructions;

[0044] The above method is implemented when the processor executes the computer-executable instructions stored in the memory.

[0045] In a fourth aspect, the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to implement the above method when executed by a processor.

[0046] In a fifth aspect, the present application also provides a computer program product, including computer execution instructions, which implement the above method when executed by a processor.

[0047] The present application provides a remote control parking method, system, computer device and storage medium. The method includes: obtaining the surrounding environment detection result of the vehicle to be controlled according to the environmental sensor; obtaining the surrounding image of the vehicle to be controlled according to the image acquisition device when the surrounding environment detection result meets the preset conditions; performing face detection according to the surrounding image to obtain the face detection result of the surrounding image; matching the face detection result with the preset face database, and when the face match is successful, performing gesture recognition on the target of the vehicle to be controlled with the successful face match to obtain the gesture recognition result; and controlling the parking of the vehicle to be controlled according to the gesture recognition result of the target of the vehicle to be controlled. The embodiment of the present application uses the environmental sensor and the image acquisition device on the vehicle to be controlled to collect the surrounding environment of the vehicle and the gestures of the target of the vehicle to be controlled, and combines technologies such as face recognition and gesture recognition to realize remote control parking of the vehicle to be controlled, and parking can be completed without carrying any mobile terminal device. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] Figure 1 A schematic diagram of an application environment for remote parking provided in one embodiment;

[0050] Figure 2 A process of a remote control parking method provided in one embodiment Figure 1 ;

[0051] Figure 3 A process of a remote control parking method provided in one embodiment Figure 2 ;

[0052] Figure 4 A process of a remote control parking method provided in one embodiment Figure 3 ;

[0053] Figure 5 A process of a remote control parking method provided in one embodiment Figure 4 ;

[0054] Figure 6 is a schematic structural diagram of a remote control parking system provided in one embodiment;

[0055] Figure 7 A schematic diagram of a system structure of a vehicle capable of completing remote parking based on gesture recognition provided in one embodiment;

[0056] Figure 8 The figure is a diagram of the internal structure of a computer device provided in one embodiment.

[0057] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0059] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0060] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0061] The remote control parking method provided in the embodiment of the present application is applied to control the parking of a vehicle and can be applied to Figure 1 In the application environment shown. Among them, the terminal detection device 102 is used to detect the surrounding environment of the vehicle to be controlled in real time, obtain the surrounding environment data and surrounding images of the vehicle to be controlled, and the controller 104 is connected to the terminal detection device 102, receives the surrounding environment data and surrounding images collected by the terminal detection device 102, and processes the data to determine whether the parking conditions are met. If the parking conditions are met, a control instruction is sent to the actuator 106, and the control instruction drives the actuator to complete the parking. The data storage system can store the data that the controller 104 needs to process. The data storage system can be integrated on the controller 104, or it can be a separate data storage device. Among them, the terminal detection device 102 can be, but is not limited to, various sensors, such as ultrasonic radars, ultrasonic sensors, ranging sensors, image acquisition devices, etc. The controller 104 can be an integrated chip.

[0062] In one embodiment, Figure 2 As shown, a remote control parking method is provided, and the method is applied to Figure 1 The controller 104 in the example is used as an example to illustrate, wherein the controller 104 is a remote control parking controller of a vehicle to be controlled, and the vehicle to be controlled includes an environmental sensor and an image acquisition device. The method includes the following steps:

[0063] Step 202, obtaining the detection result of the surrounding environment of the vehicle to be controlled according to the environmental sensor;

[0064] Among them, the environmental sensor refers to a sensor used to detect the surrounding environment of the vehicle to be controlled. Different from the image acquisition device, the environmental sensor is mainly used to obtain non-image data. The surrounding environment of the vehicle to be controlled is roughly detected through the non-image data collected by the environmental sensor, and the surrounding environment detection result of the vehicle to be controlled is determined. In actual application scenarios, for example, in order to ensure a comprehensive detection of the surrounding environment of the vehicle to be controlled, there are usually multiple environmental sensors, so the surrounding environment detection result in this step refers to the surrounding environment detection result determined based on the surrounding environment data collected by all environmental sensors on the vehicle to be controlled.

[0065] In some embodiments, the environmental sensor mainly includes a distance measuring sensor, which is used to detect obstacles around the vehicle to be controlled and the distance between the obstacles and the vehicle to be controlled. The environmental sensor can be a laser radar, an ultrasonic radar, a millimeter wave radar, an infrared sensor, etc. The location of obstacles around the vehicle to be controlled is determined by the surrounding environment data collected by the distance measuring sensor, thereby obtaining the surrounding environment detection result.

[0066] Step 204, when the surrounding environment detection result meets the preset conditions, the surrounding image of the vehicle to be controlled is acquired according to the image acquisition device;

[0067] Different from the environmental sensor, the image acquisition device is used to obtain the surrounding images of the vehicle to be controlled, and based on the surrounding images, more accurate environmental information around the vehicle to be controlled can be obtained. The image acquisition device may include but is not limited to various cameras. In one embodiment, the image acquisition device includes multiple fisheye lenses.

[0068] In actual application scenarios, there are also multiple image acquisition devices deployed around the vehicle to be controlled. It should be noted that the deployment location of the image acquisition device and the deployment location of the environmental sensor can be the same or different. Both only need to be able to fully detect the surrounding environment of the vehicle to be controlled, and the specific location is not specifically limited. The surrounding images captured by the image acquisition device are usually multiple images. In this embodiment, the surrounding images can be video streams.

[0069] The preset condition is used to determine whether there is a target around the vehicle to be controlled that wants to control the vehicle to be controlled based on the surrounding environment detection result. It can be understood that the surrounding environment detection result refers to the result output after a rough detection of the surrounding environment of the vehicle to be controlled. The surrounding environment detection result is determined by the environmental sensor. In the actual vehicle control scenario, the surrounding environment detection result can only determine whether there are obstacles around the vehicle to be controlled. Further, it can only determine whether there is a target (that is, a person) close to the vehicle to be controlled, but it is impossible to authenticate the target close to the vehicle to be controlled. Therefore, it is necessary to start the image acquisition device to collect the surrounding images of the vehicle to be controlled.

[0070] Step 206, performing face detection based on the surrounding images to obtain face detection results of the surrounding images;

[0071] The surrounding images may include multiple real-time images, and face detection is performed on the surrounding images, that is, face detection is performed on each real-time image, and then the face detection results of all real-time images are integrated to obtain the face detection results of the surrounding images. Specifically, face detection can be performed by using deep learning algorithms, such as convolutional neural networks (CNNs), to detect the faces that appear in each real-time image; then useful features are extracted from the detected faces, such as the shape, size, position and other information of the eyes, nose, mouth and other parts, to determine the face detection results of each real-time image.

[0072] It should be noted that in actual application scenarios, when deploying image acquisition devices, the field of view of adjacent image acquisition devices has a certain overlap. Therefore, the same face may appear in multiple real-time images at the same time. Therefore, after determining the face detection result of each real-time image, it is necessary to integrate the face detection results of all real-time images and remove the overlapping parts to obtain the final face detection results of the surrounding images acquired at the same time.

[0073] Step 208, matching the face detection result with a preset face database, and if the face match is successful, performing gesture recognition on the vehicle to be controlled target whose face match is successful, to obtain a gesture recognition result;

[0074] The preset face database refers to a collection of pre-stored face data of people who have the authority to control the vehicle to be controlled. The face detection result is matched with the preset face database. If the match is successful, it means that there is a target vehicle around the current vehicle to be controlled that can control the vehicle to be controlled, and the vehicle to be controlled can be parked according to the gesture of the target vehicle. Otherwise, it means that there is no target vehicle to be controlled, and there is no need to perform corresponding gesture recognition on the detected face.

[0075] When the target to be controlled is determined, the gesture of the target to be controlled can be recognized based on the image of the target to be controlled acquired by the image acquisition device to determine the gesture recognition result of the target to be controlled. Specifically, gesture recognition can be implemented by some common target detection algorithms (such as YOLO, SSD, etc.).

[0076] Step 210 , controlling the vehicle to be controlled to park according to the gesture recognition result of the target vehicle to be controlled.

[0077] Among them, parking refers to the entire process of parking a vehicle in a parking space, including operations such as parking in, parking out, driving in, and driving out, so the "controlling the parking of the vehicle to be controlled" in this step includes but is not limited to the following actions: parking the vehicle in a parking space, driving the vehicle out of a parking space, and changing to another parking space before the vehicle is parked in a parking space. In the embodiment of the present application, the meanings of different gestures are pre-set, and the corresponding parking control operation is performed on the vehicle to be controlled according to the gesture recognition result of the target vehicle to be controlled.

[0078] In the method provided in the above embodiment, the detection result of the surrounding environment of the vehicle to be controlled is obtained according to the environmental sensor; when the detection result of the surrounding environment meets the preset conditions, the surrounding image of the vehicle to be controlled is obtained according to the image acquisition device; face detection is performed according to the surrounding image to obtain the face detection result of the surrounding image; the face detection result is matched with the preset face database, and when the face match is successful, gesture recognition is performed on the target of the vehicle to be controlled with successful face matching to obtain the gesture recognition result; according to the gesture recognition result of the target of the vehicle to be controlled, the parking of the vehicle to be controlled is controlled. The embodiment of the present application uses the environmental sensor and the image acquisition device on the vehicle to be controlled to collect the surrounding environment of the vehicle and the gestures of the target of the vehicle to be controlled, and combines technologies such as face recognition and gesture recognition to realize remote control parking of the vehicle to be controlled, and parking can be completed without carrying any mobile terminal devices.

[0079] In one embodiment, if Figure 3 As shown, the environmental sensor includes an ultrasonic radar; the detection result of the surrounding environment of the vehicle to be controlled is obtained according to the environmental sensor, including:

[0080] Step 302, controlling the ultrasonic radar to start, and controlling other control modules of the vehicle to be controlled to be in a power-off state;

[0081] Step 304, obtaining surrounding environment data collected by the ultrasonic radar;

[0082] Step 306: Determine the detection result of the surrounding environment of the vehicle to be controlled according to the surrounding environment data.

[0083] In the vehicle to be controlled, all sensors on the vehicle are uniformly controlled by the controller 104, and the controller 104 can start different sensors according to actual conditions. When the vehicle is remotely parked, if the vehicle to be controlled needs to leave the parking space, the vehicle to be controlled is parked in the parking space before this, and the vehicle to be controlled is in a power-off state. The controller 104 only needs to control the ultrasonic radar to start, detect the surrounding environment through the ultrasonic radar, and obtain the surrounding environment data. After receiving the surrounding environment data, the controller 104 performs data analysis to obtain the surrounding environment detection results of the vehicle to be controlled. At this time, the corresponding control modules of other sensors of the vehicle to be controlled are in a power-off state.

[0084] In the method provided in the above embodiment, the surrounding environment of the vehicle to be controlled is first detected by ultrasonic radar to ensure that the vehicle to be controlled is in a low power consumption state. When the surrounding environment detection results meet the preset conditions, other control modules required on the vehicle to be controlled are started, which can save power of the vehicle to be controlled.

[0085] In one embodiment, the surrounding environment data includes each transmission signal of the ultrasonic radar and the echo duration of the corresponding echo signal; and determining the surrounding environment detection result of the vehicle to be controlled according to the surrounding environment data includes:

[0086] Determine the distance of the obstacle corresponding to each echo signal according to the echo duration of each transmitted signal and the corresponding echo signal;

[0087] The distances of obstacles corresponding to all echo signals are linked and analyzed to determine the surrounding environment detection results around the vehicle to be controlled.

[0088] When the vehicle to be controlled is in a low power consumption state, the ultrasonic radar emits ultrasonic waves (transmitting signals) and starts timing. It should be noted that there are multiple ultrasonic radars and multiple corresponding transmitting signals. For each transmitting signal, the transmitted ultrasonic wave propagates in the medium and will be reflected back after encountering an obstacle. The ultrasonic radar receives the echo signal and stops timing immediately to obtain the echo duration corresponding to each echo signal. The distance to the obstacle corresponding to each echo signal is calculated by the speed of the transmitted ultrasonic wave and the echo duration.

[0089] By linking multiple transmission signals from multiple ultrasonic radars to perform two-dimensional distance measurement, the geometric type and position information of the detected obstacles can be located. For example, for point / column objects and line / wall objects, ultrasonic radar can determine the shape of the object by comparing different formulas with the actual measured data.

[0090] In the method provided in the above embodiment, the detection result of the surrounding environment of the vehicle to be controlled is determined quickly and accurately through the ranging principle of the ultrasonic radar, and the ultrasonic radar has the advantage of low power consumption, which can save the power of the vehicle to be controlled.

[0091] In one embodiment, the surrounding environment detection result includes whether an obstacle is detected, the type of obstacle, the location coordinates of the obstacle, and the confidence level of the detection result; the surrounding environment detection result meets the preset conditions including:

[0092] The surrounding environment detection result includes that an obstacle is detected and the type of the obstacle is a preset obstacle type; the position coordinates of the obstacle in the surrounding environment detection result are changed and the distance between the obstacle and the vehicle to be controlled is less than the preset distance.

[0093] The type of obstacle can be determined based on the shape of the obstacle, including but not limited to pedestrians, vehicles, trash cans, trees, steps, walls, etc. The position coordinates of the obstacle are used to describe the position of the obstacle, represented by spatial coordinates, and the number of the position coordinates of the obstacle is determined based on the type of obstacle. For example, a linear obstacle can be represented by two coordinate points, and a point obstacle can be represented by one coordinate point. In one embodiment, the surrounding environment detection result also includes the moving speed of the obstacle.

[0094] Based on the above definition, in one embodiment, the surrounding environment detection result includes that an obstacle is detected and the type of the obstacle is a preset obstacle type; the position coordinates of the obstacle in the surrounding environment detection result are changed and the distance between the obstacle and the vehicle to be controlled is less than a preset distance. The preset obstacle type is a pedestrian.

[0095] In the method provided in the above embodiment, the type, distance and whether the obstacle is moving are judged through detection by the environmental sensor, and whether there are pedestrians around the vehicle to be controlled who want to control the vehicle to be controlled is quickly identified.

[0096] In one embodiment, performing face detection based on surrounding images to obtain face detection results of surrounding images includes:

[0097] Determining at least one image to be detected from surrounding images according to a preset sampling period;

[0098] Preprocessing at least one image to be detected to remove noise points in at least one image to be detected;

[0099] Perform facial feature recognition on the at least one preprocessed image to be detected to determine a face detection result of the at least one image to be detected.

[0100] The preset sampling period is used to sample a certain number of images from the surrounding images as the images to be detected. Based on the explanation in the above embodiment, the surrounding images refer to real-time image data acquired by multiple image acquisition devices. Usually, the surrounding images include video streams acquired by multiple image acquisition devices to acquire the surroundings of the vehicle to be controlled. However, face recognition does not need to be performed on every frame of the video stream. Therefore, the video stream is sampled through the preset sampling period to obtain at least one image to be detected.

[0101] Preprocessing is used to perform image processing on at least one image to be detected to improve the effectiveness of the image to be detected. Preprocessing includes but is not limited to noise reduction, white balance adjustment, distortion removal, and bad pixel removal. Based on a preset algorithm, facial feature recognition is performed on at least one preprocessed image to be detected to determine the face detection result of at least one image to be detected, thereby obtaining the face detection results of the surrounding images.

[0102] In the method provided in the above embodiment, before performing face recognition on surrounding images, the surrounding images are first sampled and preprocessed to reduce the amount of calculation for face recognition. Performing face recognition on the preprocessed images can improve the accuracy of face recognition.

[0103] In one embodiment, gesture recognition is performed on the target vehicle to be controlled whose face is successfully matched, and a gesture recognition result is obtained, including:

[0104] Acquire a gesture image of a vehicle to be controlled by an image acquisition device;

[0105] Perform gesture detection and tracking on the gesture image to determine the gesture trajectory of the target vehicle to be controlled;

[0106] According to the gesture trajectory, the gesture recognition result of the target vehicle to be controlled is determined.

[0107] Among them, the gesture image of the target to control the vehicle is obtained through an image acquisition device. It can be an image specially captured of the hand of the target to control the vehicle after adjusting the acquisition angle and range of the image acquisition device, or it can be captured from the surrounding images captured by the image acquisition device, or the surrounding images can be used directly.

[0108] In one application scenario, the image acquisition device includes four surround view cameras deployed around the vehicle to be controlled, wherein the surround view cameras are fisheye lenses. The gesture images of the target vehicle to be controlled are acquired through the four fisheye lenses, and the acquired images are preprocessed, including denoising, brightness and contrast adjustment, etc. Since the surround view camera uses a fisheye lens, significant radial distortion will occur, so image correction is required. The corrected images are then spliced ​​into a panoramic image, and gesture detection and tracking are performed on the panoramic image based on a preset target detection algorithm (such as YOLO, SSD, etc.) to obtain the gesture trajectory; the tracked gestures are recognized through a deep learning model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), and the gesture recognition result of the target vehicle to be controlled is obtained.

[0109] In the method provided in the above embodiment, the gesture of the target to be controlled is fully detected through gesture detection, tracking and recognition, and an accurate gesture recognition result is obtained.

[0110] In one embodiment, if Figure 4 As shown, according to the gesture recognition result of the target vehicle to be controlled, the vehicle to be controlled is controlled to park, including:

[0111] Step 402, analyzing the gesture recognition result of the vehicle to be controlled, and determining the command type corresponding to the gesture of the vehicle to be controlled;

[0112] Step 404, obtaining a parking function signal of the vehicle to be controlled;

[0113] Step 406 : Control the vehicle to be controlled to park according to the command type and the parking function signal.

[0114] The gesture recognition result includes the gesture type and gesture trajectory, and each gesture type has a corresponding command type. The parking function signal is used to indicate whether the vehicle to be controlled has the function of running, that is, whether the power system, transmission system, drive system, etc. of the vehicle to be controlled are fault-free. It can be understood that when the power system, transmission system, drive system, etc. of the vehicle to be controlled have faults, the vehicle cannot run.

[0115] The parking function signal is obtained through the CAN bus. When it is determined that the parking function signal indicates that the parking function of the vehicle is normal, the vehicle to be controlled can execute the control instruction corresponding to the gesture type in the gesture recognition result.

[0116] In one embodiment, if Figure 5 As shown, according to the command type and the parking function signal, the vehicle to be controlled is controlled to park, including:

[0117] Step 502, obtaining command types corresponding to all gestures of the vehicle to be controlled within a preset time period;

[0118] Step 504 , when the command types corresponding to all gestures are preset command types and the gesture parking function signal indicates that the parking function of the to-be-controlled vehicle is normal, the to-be-controlled vehicle is controlled to park according to the control instructions corresponding to the preset command types.

[0119] It should be noted that a gesture image can only determine the type of gesture of the target to be controlled at that moment, and remote parking is a continuous action. Before the vehicle completes parking, the gesture of the target to be controlled should remain unchanged. Therefore, the command types corresponding to all gestures of the target to be controlled within the preset time should be the same, or the gesture trajectory of the target to be controlled should be continuous within the preset time.

[0120] The preset command type refers to the command type that instructs the vehicle to park. There may be multiple preset command types, and each preset command type corresponds to a control instruction. That is, each gesture type corresponds to a control instruction, such as a steering instruction, an acceleration instruction, a deceleration instruction, an exterior light control instruction, etc. When the command types corresponding to all gestures are preset command types, and the gesture parking function signal indicates that the parking function of the vehicle to be controlled is normal, the vehicle to be controlled is controlled to park according to the control instruction corresponding to the preset command type.

[0121] In the method provided in the above embodiment, by detecting the command intensity corresponding to the gesture within the predicted duration, it is determined whether the gesture of the target vehicle to be controlled meets the parking conditions. The vehicle to be controlled is controlled only when the gesture of the target vehicle to be controlled is continuous, thereby avoiding multiple disordered controls of the vehicle and ensuring parking safety.

[0122] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0123] Based on the same inventive concept, an embodiment of the present application further provides a remote control parking system. The implementation solution for solving the problem provided by the system is similar to the implementation solution recorded in the above method. Therefore, the specific limitations in one or more remote control parking system embodiments provided below can refer to the limitations on the remote control parking method above, which will not be repeated here.

[0124] In one embodiment, Figure 6 As shown, a remote control parking system is provided, the system is applied to a vehicle to be controlled, the vehicle to be controlled includes an environmental sensor and an image acquisition device; the system includes a cabin-parking integrated subsystem 601 and a vehicle control subsystem 602;

[0125] The cabin-parking integrated subsystem 601 is used to control the environmental sensor and the image acquisition device, and determine the gesture recognition result of the target vehicle to be controlled corresponding to the vehicle to be controlled according to the data collected by the environmental sensor and the image acquisition device, and send the gesture recognition result to the vehicle control subsystem 602;

[0126] The vehicle control subsystem 602 is used to control the operation of the vehicle to be controlled 700 according to the received gesture recognition result.

[0127] In one embodiment, the cabin-mooring integrated subsystem 601 includes an environment detection module 6011, a face recognition module 6012 and a gesture recognition module 6013; wherein,

[0128] The environment detection module 6011 is used to obtain the detection result of the surrounding environment of the vehicle to be controlled according to the environment sensor;

[0129] The face recognition module 6012 is used to obtain the surrounding image of the vehicle to be controlled according to the image acquisition device when the surrounding environment detection result meets the preset conditions; perform face detection according to the surrounding image to obtain the face detection result of the surrounding image; and match the face detection result with the preset face database;

[0130] The gesture recognition module 6013 is used to perform gesture recognition on the vehicle to be controlled target with successful face matching when the face matching is successful, obtain the gesture recognition result, and send the gesture recognition result to the vehicle control subsystem 602.

[0131] In one of the embodiments, the cabin-parking integrated subsystem 601 is also used to obtain a status signal of the vehicle control subsystem 602 and determine whether the status signal is normal. If the status signal is abnormal, an exit control instruction is sent to the vehicle control subsystem 602.

[0132] In a specific embodiment, a vehicle capable of completing remote parking based on gesture recognition is provided. The vehicle includes the remote parking system in the above embodiment, and a controller of the system can execute the remote parking method mentioned in any of the above embodiments; the system framework of the vehicle is as follows: Figure 7 As shown, the environmental sensor includes an ultrasonic radar USS, and the image acquisition device includes a surround-view camera, wherein:

[0133] Cabin-parking integrated subsystem: uses USS for distance measurement, surround-view cameras for face recognition to complete user legal authentication, gesture recognition to complete the driver's OK / NOK gesture, and uses the parking function algorithm to control the vehicle: output lateral and longitudinal acceleration requests, corner requests, and body electronic control requests.

[0134] Vehicle control subsystem: responsible for controlling the body controller, steering actuator, acceleration actuator, and direction actuator.

[0135] The specific control logic includes: (taking remote control parking out as an example, the same applies to remote control parking in)

[0136] S1: The vehicle is in low power state, waiting to be awakened.

[0137] (1) At this time, the USS chip and MCU are powered on (the USS is always detecting the distance of objects outside the vehicle; the MCU performs energy-saving management and controls other chips to be powered off);

[0138] (2) The USS ranging process is as follows:

[0139] Transmitting ultrasonic waves: Ultrasonic radar first transmits ultrasonic waves.

[0140] Waiting for echo: Ultrasonic waves propagate in the medium and are reflected back after encountering obstacles.

[0141] Receive echo and time: The ultrasonic radar receives the reflected wave and stops timing immediately. By calculating the time difference between transmitting and receiving the ultrasonic wave, the distance to the obstacle can be calculated.

[0142] Determine obstacle type: Use multiple ultrasonic radars to perform two-dimensional distance measurement to determine the geometric type of obstacles. For example, for point / column objects and line / wall objects, ultrasonic radars can determine the shape of objects by comparing different formulas with actual measured data.

[0143] Output results: whether an obstacle is detected, the type of obstacle, the location coordinates of the obstacle, and the confidence level of its existence.

[0144] S2: When a moving object is detected approaching the vehicle and is within 5m, the USS informs the MCU of the distance and the MCU performs energy-saving management.

[0145] The MCU only wakes up the Deserializers and AVM camera; the screen subsystem (Serializer and screen) and Audio subsystem (A2B chip and external AMP, MIC, Speakers) are in an unpowered state.

[0146] S3: The four surround-view cameras start to input video streams, and at the same time, the AI ​​chip in the cabin-to-berth subsystem SOC starts to perform face recognition.

[0147] S4: After successful face recognition, the energy-saving management module of the cabin-parking integrated subsystem wakes up the remaining systems in all cabin-parking integrated subsystems that have not been awakened (it needs to wake up all components of its own domain control, and the corresponding external ECUs, screens, AMPs, MICs, and Speakers); at the same time, it needs to send network management messages to keep the vehicle control ECU awake.

[0148] S5: Start gesture recognition and complete OK / NOK recognition of gesture recognition. The specific process is as follows:

[0149] Image acquisition: The surround-view camera system collects image data of the surrounding environment in real time.

[0150] Image preprocessing: Preprocess the collected images, including denoising, brightness and contrast adjustment, etc.

[0151] Image correction: Since surround view cameras usually use fisheye lenses, significant radial distortion will occur, so image correction is required.

[0152] Image stitching: stitch the images from four cameras into a panoramic image for subsequent gesture recognition.

[0153] Gesture detection: Detect possible gestures in panoramic images, which can be achieved through some common object detection algorithms (such as YOLO, SSD, etc.).

[0154] Gesture tracking: Track detected gestures to handle continuous gesture actions.

[0155] Gesture recognition: Identify the tracked gestures, which usually requires training a deep learning model such as a convolutional neural network (CNN) or a recurrent neural network (RNN).

[0156] Action analysis: Convert the recognized gestures into specific commands or actions (OK gesture / NOK gesture).

[0157] S6: Conditions for detecting remote parking: No fault in the cabin parking subsystem, no fault in the vehicle control subsystem, and no fault in the network connection system

[0158] S7: The cabin-parking integrated subsystem starts to control the parking function: If S5 detects that the target vehicle to be controlled uses the OK gesture and the parking condition of step S6 is available, remote parking is performed. Note: The OK gesture is performed only when it is continuous. If the user uses the NOK gesture or a non-OK gesture, the parking function is suspended.

[0159] S8: When executing remote parking, the cabin-parking integrated subsystem needs to periodically send vehicle control commands (steering commands, acceleration commands, deceleration commands, and exterior lighting control commands) to the vehicle control subsystem.

[0160] S9: The cabin-parking subsystem needs to monitor the status of the vehicle control subsystem. If a vehicle control subsystem failure is detected, or the vehicle control subsystem unexpectedly executes its own request, the cabin-parking subsystem needs to issue an exit command to the vehicle control subsystem.

[0161] S10: The vehicle control subsystem controls the actuator to perform parking, parking, pausing, and exiting.

[0162] like Figure 8 As shown, an embodiment of the present application provides a computer device 800 , and the computer device 800 includes a memory 801 and a processor 802 .

[0163] The memory 801 is used to store computer-executable instructions executable by the processor;

[0164] The processor 802 implements each step of the method in the above embodiment when executing the computer execution instruction. For details, please refer to the relevant description in the above method embodiment.

[0165] Optionally, the memory 801 can be independent or integrated with the processor 802. When the memory 801 is independently provided, the computer device further includes a bus for connecting the memory 801 and the processor 802.

[0166] The embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, each step of the method in the above embodiment is implemented.

[0167] The embodiment of the present application also provides a computer program product, including computer executable instructions, which implement the various steps of the method in the above embodiment when executed by a processor.

[0168] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer execution instructions in the computer program are executed, they can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0169] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A remote control parking method, characterized in that: Applied to a vehicle to be controlled, the vehicle to be controlled includes an environmental sensor and an image acquisition device, and the method includes: Acquiring a detection result of the surrounding environment of the vehicle to be controlled according to the environmental sensor; When the surrounding environment detection result meets the preset conditions, acquiring the surrounding image of the vehicle to be controlled according to the image acquisition device; Performing face detection based on the surrounding images to obtain face detection results of the surrounding images; Matching the face detection result with a preset face database, and if the face match is successful, performing gesture recognition on the target vehicle to be controlled whose face match is successful, to obtain a gesture recognition result; The vehicle to be controlled is controlled to park according to the gesture recognition result of the vehicle to be controlled.

2. The method according to claim 1, characterized in that The environmental sensor includes an ultrasonic radar; and obtaining the detection result of the surrounding environment of the vehicle to be controlled according to the environmental sensor includes: Controlling the ultrasonic radar to start, and controlling other control modules of the vehicle to be controlled to be in a power-off state; Acquiring surrounding environment data collected by the ultrasonic radar; A detection result of the surrounding environment of the vehicle to be controlled is determined according to the surrounding environment data.

3. The method according to claim 2, characterized in that The surrounding environment data includes each transmission signal of the ultrasonic radar and the echo duration of the corresponding echo signal; the surrounding environment detection result of the vehicle to be controlled is determined according to the surrounding environment data, including: Determine the distance of the obstacle corresponding to each echo signal according to the echo duration of each transmitted signal and the corresponding echo signal; The distances of obstacles corresponding to all echo signals are linked and analyzed to determine the surrounding environment detection results around the vehicle to be controlled.

4. The method according to any one of claims 1 to 3, characterized in that The surrounding environment detection result includes whether an obstacle is detected, the type of obstacle, the location coordinates of the obstacle and the confidence level of the detection result; The surrounding environment detection result meets the preset conditions including: The surrounding environment detection result includes that an obstacle is detected and the type of the obstacle is a preset obstacle type; The position coordinates of the obstacle in the surrounding environment detection result change and the distance between the obstacle and the vehicle to be controlled is less than a preset distance.

5. The method according to claim 1, characterized in that The method of controlling the vehicle to be controlled to park according to the gesture recognition result of the vehicle to be controlled includes: Analyze the gesture recognition result of the target vehicle to be controlled, and determine the command type corresponding to the gesture of the target vehicle to be controlled; Acquiring a parking function signal of the vehicle to be controlled; The vehicle to be controlled is controlled to park according to the command type and the parking function signal.

6. The method according to claim 5, characterized in that The controlling the vehicle to be controlled to park according to the command type and the parking function signal includes: Obtaining command types corresponding to all gestures of the vehicle control target within a preset time period; When the command types corresponding to all the gestures are preset command types and the gesture parking function signal indicates that the parking function of the to-be-controlled vehicle is normal, the to-be-controlled vehicle is controlled to park according to the control instructions corresponding to the preset command types.

7. The method according to claim 1, characterized in that The performing face detection according to the surrounding images to obtain face detection results of the surrounding images includes: Determining at least one image to be detected from the surrounding images according to a preset sampling period; Preprocessing the at least one image to be detected to remove noise in the at least one image to be detected; Perform facial feature recognition on the at least one preprocessed image to be detected to determine a face detection result of the at least one image to be detected.

8. A remote control parking system, characterized in that: The system is applied to a vehicle to be controlled, and the vehicle to be controlled includes an environmental sensor and an image acquisition device; the system includes a cabin-parking integrated subsystem and a vehicle control subsystem; The cabin-parking integrated subsystem is used to control the environmental sensor and the image acquisition device, and determine the gesture recognition result of the target vehicle to be controlled corresponding to the vehicle to be controlled according to the data collected by the environmental sensor and the image acquisition device, and send the gesture recognition result to the vehicle control subsystem; The vehicle control subsystem is used to control the operation of the vehicle to be controlled according to the received gesture recognition result.

9. A computer device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images