Car window positioning method and device and storage medium

Abstract

The invention discloses a vehicle window positioning method and device and a storage medium, and relates to the technical field of image recognition, and the method comprises the steps: obtaining at least one group of vehicle side views of a current vehicle; carrying out preset vehicle key part detection on the at least one group of vehicle side views to obtain a key part detection result; determining coordinate information of a preset vehicle key part according to camera parameters of the vehicle side view and the key part detection result; and target vehicle window position information is determined according to the preset vehicle window offset information of the preset vehicle key part and the coordinate information of the preset vehicle key part. According to the method, the coordinate information of the preset vehicle key part is recognized, the vehicle window position information is obtained by combining the preset vehicle window deviation information of the preset vehicle key part, errors and interference possibly caused by direct contact with the vehicle window are avoided in a non-contact measurement mode, and the accuracy of the obtained vehicle window position information is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle window positioning method, device, and storage medium. Background Art

[0002] In vehicle import and export scenarios such as border crossings, expressways, and parking lots, vehicles need to undergo security inspections to ensure the safety of both passengers and vehicles. To improve the efficiency of vehicle import and export, existing methods have proposed passenger-free customs inspection equipment, including inspection terminals and actuators (such as robotic arms). The actuator drives the inspection terminal to move in multiple directions, allowing the inspection terminal to accurately align with the windows of the vehicle passing through customs, enabling rapid information verification of the vehicle and personnel.

[0003] Currently, non-contact sensors such as ultrasonic and laser sensors are commonly used to determine vehicle window positions. However, vehicle window glass is highly reflective, transparent, and easily affected by ambient light. Therefore, direct measurement of window glass using these non-contact sensors can easily lead to deviations in the measured window position, which in turn affects vehicle inspection efficiency. Summary of the Invention

[0004] The main purpose of this application is to provide a vehicle window positioning method, device and storage medium, aiming to solve the technical problem that when the existing non-contact sensor is used to directly measure the vehicle window glass, the measured vehicle window position is easily deviated.

[0005] To achieve the above objectives, the present application proposes a vehicle window positioning method, which includes:

[0006] Obtain at least one set of vehicle side views of the current vehicle;

[0007] Performing a preset vehicle key part detection on at least one set of the vehicle side views to obtain a key part detection result;

[0008] Determining coordinate information of preset key parts of the vehicle according to the camera parameters of the vehicle side view and the key part detection results;

[0009] The target window position information is determined according to the preset window offset information of the preset key part of the vehicle and the coordinate information of the preset key part of the vehicle.

[0010] In one embodiment, a group of the vehicle side views includes a first viewpoint image and a second viewpoint image, and the step of performing preset vehicle key part detection on at least one group of the vehicle side views to obtain key part detection results includes:

[0011] Performing preset vehicle key part detection on the first viewpoint image and the second viewpoint image respectively to obtain a first view detection result and a second view detection result;

[0012] The first view detection result and the second view detection result are respectively compared with a preset number of key parts of the vehicle, and a key part detection result is determined according to the comparison result.

[0013] In one embodiment, the step of determining the coordinate information of the preset key parts of the vehicle based on the camera parameters of the vehicle side view and the key part detection results includes:

[0014] Obtaining depth information according to the first viewpoint image, the second viewpoint image, and corresponding camera parameters;

[0015] The coordinate information corresponding to the preset key part of the vehicle is obtained according to the depth information and the key part detection result.

[0016] In one embodiment, the step of obtaining depth information according to the first viewpoint image, the second viewpoint image, and the corresponding camera parameters includes:

[0017] extracting feature points corresponding to the first viewpoint image and the second viewpoint image respectively, matching the feature points according to a preset feature point matching algorithm, and obtaining disparity information based on the matching results;

[0018] Depth information is obtained according to the disparity information and camera parameters corresponding to the first viewpoint image and the second viewpoint image respectively.

[0019] In one embodiment, the step of determining the target window position information based on the preset window offset information of the preset key vehicle part and the coordinate information of the preset key vehicle part includes:

[0020] Screening the preset key vehicle parts to obtain candidate key vehicle parts;

[0021] A target vehicle key part is determined from the candidate vehicle key parts, and target vehicle window position information is determined according to preset vehicle window offset information of the target vehicle key part and coordinate information of the target vehicle key part.

[0022] In one embodiment, the step of determining a target vehicle key part from the candidate vehicle key parts and determining target vehicle window position information based on preset vehicle window offset information of the target vehicle key part and coordinate information of the target vehicle key part includes:

[0023] determining a target vehicle key part from among the candidate vehicle key parts based on a preset priority order;

[0024] Acquiring preset window offset information of a key part of the target vehicle, wherein the preset window offset information is a preset position offset between the key part of the target vehicle and the target window;

[0025] Based on the coordinate information of the key parts of the target vehicle, coordinate conversion is performed according to the position offset to obtain the coordinate information of the target vehicle window and determine it as the target vehicle window position information.

[0026] In one embodiment, when the acquired vehicle side views are a group, the preset vehicle key parts are screened according to a first preset screening strategy;

[0027] The first preset screening strategy is a strategy for screening the preset key vehicle parts according to the preset relative positions and the coordinate information of the preset key vehicle parts.

[0028] In one embodiment, when the acquired vehicle side views are at least two groups, the preset vehicle key parts are screened according to the first preset screening strategy and / or the second preset screening strategy;

[0029] Wherein, the first preset screening strategy is a strategy for screening the preset key vehicle parts according to the preset relative position and the coordinate information of the preset key vehicle parts;

[0030] The second preset screening strategy is a strategy for screening the preset key vehicle parts based on a preset difference threshold and a coordinate difference. The coordinate difference is determined by the converted coordinates of the preset key vehicle parts in the standard coordinate system corresponding to the coordinate information of each vehicle side view. The standard coordinate system is determined based on the installation perspective information corresponding to the vehicle side view.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a vehicle window positioning device, which includes: a memory, a processor, and a vehicle window positioning program stored in the memory and executable on the processor, wherein the vehicle window positioning program is configured to implement the steps of the vehicle window positioning method described above.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium and stores a vehicle window positioning program. When the vehicle window positioning program is executed by the processor, the steps of the vehicle window positioning method described above are implemented.

[0033] The present application provides a vehicle window positioning method, comprising: obtaining at least one set of vehicle side views of the current vehicle; performing a preset vehicle key part detection on at least one set of the vehicle side views to obtain key part detection results; determining coordinate information of the preset vehicle key part based on camera parameters of the vehicle side views and the key part detection results; and determining target vehicle window position information based on preset window offset information of the preset vehicle key part and the coordinate information of the preset vehicle key part. The present application obtains vehicle window position information by identifying the coordinate information of the preset vehicle key part and combining it with the preset window offset information of the preset vehicle key part. The non-contact measurement method avoids errors and interference that may be caused by direct contact with the window, thereby improving the accuracy of the obtained window position information. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 This is a flow chart of the first embodiment of the vehicle window positioning method of the present application;

[0037] Figure 2 This is a schematic diagram of the binocular camera installation position;

[0038] Figure 3 It is a schematic diagram for presetting the key parts detection of the vehicle;

[0039] Figure 4 This is a flow chart of a second embodiment of the vehicle window positioning method of the present application;

[0040] Figure 5 This is a flow chart of a third embodiment of the vehicle window positioning method of the present application;

[0041] Figure 6 This is a structural diagram of the vehicle window positioning device of this application.

[0042] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0044] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0045] The embodiment of the present application provides a vehicle window positioning method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of the vehicle window positioning method of the present application. In this embodiment, the method includes steps S10 to S40:

[0046] Step S10: Acquire at least one set of vehicle side views of the current vehicle.

[0047] It should be noted that the method of this embodiment can be applied in scenarios of vehicle clearance security inspections, and can also be applied in other scenarios where window position information needs to be obtained. The execution subject of the method of this embodiment can be a computing electronic device with data processing, program execution, and network communication functions, such as the master control platform and robotic arm control module in the passenger clearance inspection system without getting off the vehicle, or other vehicle window positioning devices that can achieve the same or similar functions. This embodiment uses the master control platform (hereinafter referred to as the "master control platform") in the passenger clearance inspection system without getting off the vehicle as an example to illustrate this embodiment and the following embodiments.

[0048] It should be understood that the passenger clearance inspection system without getting off the vehicle can also include a passenger clearance inspection device without getting off the vehicle (hereinafter referred to as "inspection equipment") arranged on the side of the vehicle passage. The inspection equipment is equipped with at least one set of binocular cameras (each set of binocular cameras can capture a side view of the vehicle from one perspective) for collecting side views of the vehicle from different perspectives and sending them to the master control platform when the vehicle travels to the inspection area in the vehicle passage.

[0049] It should also be noted that the binocular camera can include two black-and-white cameras with different viewpoints and an infrared projector. The infrared projector is used to project an invisible fixed infrared texture pattern (such as speckle or stripes) into the scene to enhance the image information richness of the vehicle side view obtained in environments with less obvious textures. In another optional embodiment, the binocular camera can also include at least one full-color camera to assist in the vehicle color verification requirements of the passenger-free customs inspection system.

[0050] You can refer to here Figure 2 This section explains the installation example of a binocular camera. Figure 2 This is a schematic diagram of the binocular camera installation position. Figure 2 As shown in 2-A: The inspection equipment is arranged on one side or both sides of the inspection area, and the inspection equipment may be provided with at least one set of binocular cameras.

[0051] In one embodiment, two sets of binocular cameras can be installed on the inspection equipment on each side; Figure 2 As shown in Figure 2-B: For the inspection equipment on one side, the two sets of binocular cameras can be installed at the upper side position and the upper position respectively to capture the side view of the vehicle from different perspectives.

[0052] In the specific implementation, taking a group of binocular cameras installed on the inspection equipment as an example, when the current vehicle drives to the inspection area in the vehicle channel, the master control platform can obtain the vehicle side view captured by a group of binocular cameras on the inspection equipment on one side.

[0053] Step S20: performing a preset vehicle key part detection on at least one group of the vehicle side views to obtain a key part detection result.

[0054] It should be understood that the preset key vehicle parts can be body structure components located on the side of the vehicle that are predetermined by the user, such as rearview mirrors, door handles, B-pillars, etc. The relative positions of these body structure components and the windows are fixed.

[0055] It should be noted that computer vision-based target detection technology can be used to detect key vehicle parts. This technology is used to identify specific targets in images or videos and determine their locations. By training deep learning models such as convolutional neural networks (CNNs), it can accurately detect a variety of targets.

[0056] For example, an object detection model suitable for detecting preset key vehicle parts can be initialized based on an existing object detection framework (such as YOLOv8). This model is then trained and optimized using a large number of vehicle side views and corresponding annotations of preset key vehicle parts, ultimately yielding an optimized object detection model. This model can handle complex scenes and adapt to varying lighting and posture changes, thereby improving the accuracy of detection results.

[0057] In the specific implementation, the target detection model is used to detect the preset key parts of the vehicle on the side view. The key part detection results are represented as several detection boxes on the side view of the vehicle with the preset key part names and probability labels, such as Figure 3 As shown, Figure 3 This is a schematic diagram for presetting the detection of key vehicle parts.

[0058] Furthermore, since a set of vehicle side views is captured by a set of binocular cameras, and a set of binocular cameras can simultaneously capture vehicle side views corresponding to two viewpoints, the vehicle side views can include a first viewpoint image and a second viewpoint image. In order to determine the key part detection results, step S20 specifically includes: steps S201 to S202:

[0059] Step S201: performing preset vehicle key part detection on the first viewpoint image and the second viewpoint image respectively to obtain a first view detection result and a second view detection result.

[0060] In a specific implementation, the first viewpoint image and the second viewpoint image may be respectively input into the above-mentioned target detection model to obtain a first view detection result and a second view detection result.

[0061] Step S202: comparing the first view detection result and the second view detection result respectively with respect to a preset number of key vehicle parts, and determining a key part detection result according to the comparison result.

[0062] It should be noted that due to the different installation positions of the two cameras in the binocular camera, the image content of the two viewpoint images (the first viewpoint image and the second viewpoint image) captured by them is different, that is, the number of detection frames in the first view detection result and the second view detection result and the corresponding preset key part name labels may be different.

[0063] In a specific implementation, the first view detection result and the second view detection result can be compared and the view detection result with the greater number of detection frames can be selected as the key part detection result. Alternatively, if the number of detection frames in the first view detection result and the second view detection result is the same, the view detection result with the higher probability of corresponding detection frames can be selected as the key part detection result.

[0064] Step S30: determining coordinate information of preset key parts of the vehicle according to the camera parameters of the vehicle side view and the key part detection results.

[0065] It should be understood that the camera parameters of the vehicle side view may include the intrinsic parameters (focal length, optical center, etc.) and extrinsic parameters (position, attitude, etc.) of the binocular camera and the baseline distance of the binocular camera, which can be obtained by pre-calibrating the binocular camera.

[0066] It should be noted that based on the above-mentioned key parts detection results, the two-dimensional coordinates of each preset vehicle key part can be determined, and then the above-mentioned camera parameters can be used to convert the two-dimensional coordinates of each preset vehicle key part into three-dimensional space coordinates, that is, to determine the coordinate information of the preset vehicle key part.

[0067] Step S40: determining target window position information according to the preset window offset information of the preset key part of the vehicle and the coordinate information of the preset key part of the vehicle.

[0068] It should be noted that the preset window offset information is the pre-set positional offset between the preset key vehicle part and the target window of the current vehicle. For example, the preset window offset information is set as a predetermined value in centimeters or millimeters. This position offset can be an offset in a standard coordinate system determined based on the binocular camera's installation view angle information: in the standard coordinate system, the offset between the center of the detection frame corresponding to the preset key vehicle part and the center of the detection frame of the target window can be expressed as (Δx, Δy, Δz).

[0069] In the specific implementation, taking the door handle as an example of a key part of the vehicle, we can first determine whether the coordinate information of the door handle is a three-dimensional coordinate in the standard coordinate system; if so, we obtain the preset window offset information corresponding to the door handle; then, according to the three-dimensional coordinates of the door handle (x dh ,y dh , z dh ) and its corresponding preset window offset information (Δx, Δy, Δz), and obtain the three-dimensional coordinates of the target window (x dh +Δx,y dh +Δy,z dh +Δz).

[0070] This embodiment obtains window position information by identifying the coordinate information of preset key parts of the vehicle, and then combines it with the preset window offset information of the preset key parts of the vehicle. Compared with the existing method, the non-contact measurement method avoids the errors and interference that may be caused by direct contact with the window, thereby improving the accuracy of the obtained window position information.

[0071] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 4 , Figure 4 This is a flow chart of the second embodiment of the vehicle window positioning method of the present application.

[0072] In this embodiment, in order to specifically illustrate the process of determining the coordinate information of the preset key parts of the vehicle, step S30 specifically includes: steps S301 to S302:

[0073] Step S301: obtaining depth information according to the first viewpoint image, the second viewpoint image, and corresponding camera parameters.

[0074] It should be understood that since the first and second viewpoint images are images of the vehicle captured at the same time and in the same scene, they may contain several matching feature points. Based on these matching feature points, the binocular camera's disparity information can be calculated, i.e., the difference in horizontal position of the same object in the first and second viewpoint images. This disparity information can then be combined with camera parameters to calculate depth information using image triangulation techniques.

[0075] It should be noted that image triangulation technology is a method of determining the position of an object in three-dimensional space using images from multiple viewpoints. It obtains three-dimensional information from two-dimensional image data by comparing the positions of feature points of the same object in images from different viewpoints. Therefore, step S301 specifically includes steps S3011 to S3012:

[0076] Step S3011: extracting feature points corresponding to the first viewpoint image and the second viewpoint image respectively, matching the feature points according to a preset feature point matching algorithm, and obtaining disparity information according to the matching results.

[0077] It should be understood that a feature point extraction algorithm can be used to extract feature points from the first viewpoint image and the second viewpoint image respectively. Common feature point extraction algorithms can be any algorithm for extracting feature points, such as: Fast Corner Detection Algorithm (FAST), Scale-Invariant Feature Transform Algorithm (SIFT), and Speeded Up Robust Feature Algorithm (SURF).

[0078] Specifically, the feature point extraction algorithm can detect key feature points (such as corner points, edge points) in the first viewpoint image and the second viewpoint image respectively, and generate a descriptor for each key feature point for subsequent matching.

[0079] It should be noted that the feature point matching algorithm used to match each feature point can be the brute force matching algorithm (BFMatcher) or the fast nearest neighbor search algorithm (FLANNMatcher). Since disparity information is the horizontal position difference of the same object in two images, for each matched feature point pair, the horizontal coordinate difference between the first viewpoint image and the second viewpoint image can be calculated and determined as the disparity corresponding to the binocular camera.

[0080] Step S3012: obtaining depth information according to the disparity information and camera parameters corresponding to the first viewpoint image and the second viewpoint image respectively.

[0081] It should be understood that the camera parameters may be the focal length of the binocular camera and the baseline distance of the binocular camera (the horizontal distance between the optical centers of the two cameras). According to the parallax and camera parameters of the binocular camera, the depth information may be calculated using the following depth calculation formula:

[0082]

[0083] Where Z represents the depth, f represents the camera focal length, B represents the baseline distance, and d represents the parallax.

[0084] It should be understood that based on the above depth calculation formula, a depth map corresponding to the depth information of the first viewpoint image and the second viewpoint image can be obtained, and the depth map includes depth values corresponding to any feature points.

[0085] Step S302: Obtaining coordinate information corresponding to the preset vehicle key parts according to the depth information and the key part detection result.

[0086] It should be noted that for each detected preset key part of the vehicle, the corresponding depth value can be extracted from the depth map according to the position of the preset key part of the vehicle and its corresponding detection frame as the depth coordinate value corresponding to the preset key part of the vehicle.

[0087] In addition, in order to save computational effort, when performing feature point matching in the aforementioned depth information calculation process, only the feature points in each detection frame can be matched to obtain disparity information, thereby directly obtaining the depth value corresponding to the position of each detection frame based on the disparity information.

[0088] It should be understood that the coordinate information corresponding to the preset key part of the vehicle is the three-dimensional coordinate (X, Y, Z) of the preset key part of the vehicle, and the three-dimensional coordinate can be calculated by the following formula:

[0089]

[0090] Among them, (u, v) is the two-dimensional coordinate of the key parts of the vehicle preset in the key parts detection result (second view detection result); K is the camera internal parameter matrix, which includes the focal length f of the binocular camera and the optical center coordinates of the camera corresponding to the second viewpoint image (c x , c y ).

[0091] In a specific implementation, the two-dimensional coordinates of the center of the vehicle key part detection frame can be preset to represent the two-dimensional coordinates of the preset vehicle key part, and then combined with the depth value calculated based on the feature point of the center of the frame, the three-dimensional coordinates of the preset vehicle key part can be finally determined.

[0092] This embodiment uses the first viewpoint image and the second viewpoint image to perform feature point matching and depth information calculation, which can effectively reduce the errors that may be caused by a single viewpoint, and then uses image triangulation technology to extract the accurate three-dimensional coordinates of each preset vehicle key part from the two-dimensional image, which is conducive to improving the reliability of the target window position information obtained subsequently.

[0093] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the first and second embodiments can be referred to above and will not be described in detail. Figure 5 , Figure 5 This is a flow chart of the third embodiment of the vehicle window positioning method of the present application.

[0094] In this embodiment, considering that the number of preset key vehicle parts is not unique, a unique target key vehicle part can be determined from the preset key vehicle parts as a reference point for window positioning. Therefore, step S40 specifically includes: steps S401 to S402:

[0095] Step S401: Screening the preset key vehicle parts to obtain candidate key vehicle parts.

[0096] It should be understood that the vehicle side views obtained above may be one or more groups. In the case of multiple groups, steps S10 to S30 may be applied to each group of vehicle side views to obtain the three-dimensional coordinates of the preset key vehicle parts within that group of vehicle side views. Furthermore, multiple preset key vehicle parts may be detected based on each group of vehicle side views. Therefore, a corresponding preset screening strategy may be employed to screen each preset key vehicle part based on the number of vehicle side views.

[0097] When the acquired vehicle side views are a group, the preset vehicle key parts can be screened according to a first preset screening strategy. The first preset screening strategy is a strategy for screening the preset vehicle key parts according to the preset relative positions and the coordinate information of the preset vehicle key parts.

[0098] Specifically, the three-dimensional coordinates of each preset vehicle key part in the side view of the vehicle (regarded as the three-dimensional coordinates in the standard coordinate system) can be obtained first, and then the three-dimensional coordinates of each preset vehicle key part can be judged whether they meet the pre-set relative position relationship of each preset vehicle key part. After screening out the preset vehicle key parts that do not meet the relative position relationship, the alternative vehicle key parts are obtained.

[0099] For example, if in the preset relative position relationship: the rearview mirror is located in front of the B-pillar (expressed in the coordinates as the three-dimensional coordinates of the rearview mirror are closer to the origin in the x-axis direction than the B-pillar); and the three-dimensional coordinates of the rearview mirror determined by the above steps are farther away from the origin in the x-axis direction than the three-dimensional coordinates of the B-pillar, then the two preset vehicle key parts of the rearview mirror and the B-pillar are screened out.

[0100] When at least two groups of vehicle side views are obtained, the preset key vehicle parts can be screened according to a first preset screening strategy and / or a second preset screening strategy. The first preset screening strategy is a strategy for screening the preset key vehicle parts based on a preset relative position and coordinate information of the preset key vehicle parts; the second preset screening strategy is a strategy for screening the preset key vehicle parts based on a preset difference threshold and a coordinate difference, where the coordinate difference is determined by the converted coordinates of the preset key vehicle parts in a standard coordinate system corresponding to the coordinate information of each vehicle side view, and the standard coordinate system is determined based on the installation perspective information corresponding to the vehicle side view.

[0101] The process of screening using the first preset screening strategy is as described above and will not be repeated here. The process of screening using the second preset screening strategy is specifically as follows:

[0102] First, a standard coordinate system can be determined based on the installation perspective information corresponding to each vehicle side view; then, based on the standard coordinate system, the coordinate information of the preset vehicle key parts in each vehicle side view is converted to obtain the corresponding converted coordinates; finally, the coordinate difference is obtained based on each converted coordinate; the coordinate difference is compared with the preset difference threshold, and the preset vehicle key parts corresponding to the coordinate difference greater than the preset difference threshold are screened out to obtain the alternative vehicle key parts.

[0103] It should be noted that the installation angle information corresponding to each vehicle side view is the installation angle of the binocular camera group to which each vehicle side view belongs. Based on these installation angles, a standard coordinate axis centered on the current vehicle can be established. For example, with a fixed point on the current vehicle as the origin, the vehicle's length is the x-axis, the width is the y-axis, and the height is the z-axis.

[0104] Specifically, the three-dimensional coordinates of the preset key parts of the vehicle in each vehicle side view are uniformly converted to the standard coordinate system using a coordinate transformation matrix (which can be calculated based on the installation perspective) to obtain the converted coordinates. Then in the standard coordinate system, the same preset key part of the vehicle can correspond to two different three-dimensional coordinates (converted coordinates). Then the coordinate difference between the two three-dimensional coordinates corresponding to the same preset key part of the vehicle is calculated. The coordinate difference can be the Euclidean distance or the distance difference in a specific direction. Finally, it is determined whether the coordinate difference is greater than a preset difference threshold. If it is greater, it means that the position difference of the preset key part of the vehicle in different vehicle side views is large, and it may not be suitable for subsequent applications, so it is screened out.

[0105] It should also be noted that when at least two groups of vehicle side views are obtained, when the preset vehicle key parts are screened according to the first preset screening strategy and the second preset screening strategy, this embodiment does not limit the order of using these two preset screening strategies.

[0106] In a specific implementation, based on the number of vehicle side views, the preset vehicle key parts can be screened according to the first preset screening strategy and / or the second preset screening strategy to obtain alternative vehicle key parts and the three-dimensional coordinates of the alternative vehicle key parts in the standard coordinate system.

[0107] Step S402: determining a target vehicle key part from the candidate vehicle key parts, and determining target vehicle window position information according to preset vehicle window offset information of the target vehicle key part and coordinate information of the target vehicle key part.

[0108] It should be noted that, considering that the number of candidate vehicle key parts obtained through the above screening may not be unique, a unique target vehicle key part may be determined from the candidate vehicle key parts according to a preset priority order. Step S402 specifically includes steps S4021 to S4023:

[0109] Step S4021: determining a target vehicle key part from among the candidate vehicle key parts based on a preset priority order.

[0110] It can be understood that the preset priority order can be customized according to the different categories of the current vehicle, for example, set to B-pillar>rearview mirror>door handle, or set to door handle>B-pillar>rearview mirror; in addition, the preset priority order can also be generated according to the relative position of each key part of the vehicle and the target window in the current vehicle, that is, the key part of the vehicle that is closer to the target window is given a higher priority.

[0111] In a specific implementation, the preset priority order of the corresponding category can be determined according to the vehicle category of the current vehicle, and then the alternative vehicle key part with the highest priority can be determined as the target vehicle key part among the alternative vehicle key parts according to the preset priority order.

[0112] For example, if the rearview mirror has the highest priority in the set priority order and there is a rearview mirror among the alternative vehicle key parts, the rearview mirror is used as the target vehicle key part and other alternative vehicle key parts are screened out; or, the distance between each alternative vehicle key part and the target window is directly compared, and the alternative vehicle key part with the closest distance is determined as the target vehicle key part.

[0113] Step S4022: Obtain preset window offset information of key parts of the target vehicle.

[0114] It should be noted that the preset window offset information is a preset position offset between a key part of the target vehicle and a target window.

[0115] Step S4023: Based on the coordinate information of the key parts of the target vehicle, coordinate conversion is performed according to the position offset to obtain the coordinate information of the target window and determine it as the target window position information.

[0116] In the specific implementation, the three-dimensional coordinates of the key parts of the target vehicle are used as a reference, and coordinate transformation is performed on this basis according to the position offset to obtain the three-dimensional coordinates of the target window. Therefore, control instruction information can be generated based on the three-dimensional coordinates of the target window to realize the control of the robotic arm in the vehicle clearance security inspection scenario.

[0117] In this embodiment, taking into account that the vehicle side views obtained above can be one group or multiple groups, and the number of preset vehicle key parts detected in each group of vehicle side views is not unique, a screening strategy based on the number of vehicle side view groups is first adopted to ensure the accuracy of the three-dimensional coordinates of the obtained alternative vehicle key parts; then, a unique target vehicle key part is determined among the alternative vehicle key parts according to the pre-set priority order, thereby ensuring the reliability of the three-dimensional coordinates of the target vehicle window obtained by coordinate transformation based on the target vehicle key part.

[0118] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the vehicle window positioning method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0119] The present application provides a vehicle window positioning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the vehicle window positioning method in the above-mentioned embodiment 1.

[0120] Reference below Figure 6 , Figure 6 The following is a schematic diagram of the structure of the vehicle window positioning device of the present application. The vehicle window positioning device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The vehicle window positioning device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0121] like Figure 6 As shown, the vehicle window positioning device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the vehicle window positioning device. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems may be connected to the input / output interface 1006: an input device 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape or hard disk; and a communication device 1009. The communication device 1009 can allow the vehicle window positioning device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a vehicle window positioning device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0122] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0123] The vehicle window positioning device provided in this application utilizes the vehicle window positioning method of the aforementioned embodiment to solve the technical problem of vehicle window positioning. Compared to the prior art, the beneficial effects of the vehicle window positioning device provided in this application are the same as those of the vehicle window positioning method provided in the aforementioned embodiment. Other technical features of the vehicle window positioning device are the same as those disclosed in the aforementioned embodiment and are not further described here.

[0124] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0125] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0126] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer program) stored thereon, and the computer-readable program instructions are used to execute the vehicle window positioning method in the above embodiment.

[0127] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0128] The computer-readable storage medium may be included in the vehicle window positioning device; or may exist independently without being assembled into the vehicle window positioning device.

[0129] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the vehicle window positioning device, the vehicle window positioning device is enabled to: obtain at least one set of vehicle side views of the current vehicle; perform preset vehicle key part detection on the vehicle side view by the vehicle window positioning to obtain key part detection results; determine the coordinate information of the preset vehicle key parts based on the camera parameters of the vehicle side view and the key part detection results; and determine the target window position information based on the preset window offset information of the preset vehicle key parts and the coordinate information of the preset vehicle key parts.

[0130] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0131] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned vehicle window positioning method, thereby resolving the technical problem of vehicle window positioning. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the vehicle window positioning method provided in the aforementioned embodiment, and are not further elaborated here.

[0132] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional elements in the process, method, article, or system comprising the element.

[0133] The serial numbers of the above-mentioned embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments. Moreover, they are only some embodiments of the present application and do not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the description and drawings of the present application under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A vehicle window positioning method, characterized in that: The method comprises: Obtain at least one set of vehicle side views of the current vehicle; Performing a preset vehicle key part detection on at least one set of the vehicle side views to obtain a key part detection result; Determining coordinate information of preset key parts of the vehicle according to the camera parameters of the vehicle side view and the key part detection results; The target window position information is determined according to the preset window offset information of the preset key part of the vehicle and the coordinate information of the preset key part of the vehicle.

2. The method according to claim 1, wherein A set of the vehicle side views includes a first viewpoint image and a second viewpoint image, and the step of performing preset vehicle key part detection on at least one set of the vehicle side views to obtain key part detection results includes: Performing preset vehicle key part detection on the first viewpoint image and the second viewpoint image respectively to obtain a first view detection result and a second view detection result; The first view detection result and the second view detection result are respectively compared with a preset number of key parts of the vehicle, and a key part detection result is determined according to the comparison result.

3. The method according to claim 2, wherein The step of determining the coordinate information of the preset key parts of the vehicle based on the camera parameters of the vehicle side view and the key part detection results includes: Obtaining depth information according to the first viewpoint image, the second viewpoint image, and corresponding camera parameters; The coordinate information corresponding to the preset key part of the vehicle is obtained according to the depth information and the key part detection result.

4. The method according to claim 3, wherein The step of obtaining depth information according to the first viewpoint image, the second viewpoint image, and the corresponding camera parameters includes: extracting feature points corresponding to the first viewpoint image and the second viewpoint image respectively, matching the feature points according to a preset feature point matching algorithm, and obtaining disparity information based on the matching results; Depth information is obtained according to the disparity information and camera parameters corresponding to the first viewpoint image and the second viewpoint image respectively.

5. The method according to claim 1, wherein The step of determining the target window position information based on the preset window offset information of the preset key part of the vehicle and the coordinate information of the preset key part of the vehicle includes: Screening the preset key vehicle parts to obtain candidate key vehicle parts; A target vehicle key part is determined from the candidate vehicle key parts, and target vehicle window position information is determined according to preset vehicle window offset information of the target vehicle key part and coordinate information of the target vehicle key part.

6. The method according to claim 5, wherein The step of determining the target vehicle key part from the candidate vehicle key parts, and determining the target vehicle window position information based on the preset vehicle window offset information of the target vehicle key part and the coordinate information of the target vehicle key part, includes: determining a target vehicle key part from among the candidate vehicle key parts based on a preset priority order; Acquiring preset window offset information of a key part of the target vehicle, wherein the preset window offset information is a preset position offset between the key part of the target vehicle and the target window; Based on the coordinate information of the key parts of the target vehicle, coordinate conversion is performed according to the position offset to obtain the coordinate information of the target vehicle window and determine it as the target vehicle window position information.

7. The method according to any one of claims 1 to 6, characterized in that When the acquired vehicle side views are a group, screening the preset key parts of the vehicle according to a first preset screening strategy; The first preset screening strategy is a strategy for screening the preset key vehicle parts according to the preset relative positions and the coordinate information of the preset key vehicle parts.

8. The method according to any one of claims 1 to 6, characterized in that When the acquired vehicle side views are at least two groups, screening the preset vehicle key parts according to the first preset screening strategy and / or the second preset screening strategy; Wherein, the first preset screening strategy is a strategy for screening the preset key vehicle parts according to the preset relative position and the coordinate information of the preset key vehicle parts; The second preset screening strategy is a strategy for screening the preset key vehicle parts based on a preset difference threshold and a coordinate difference. The coordinate difference is determined by the converted coordinates of the preset key vehicle parts in the standard coordinate system corresponding to the coordinate information of each vehicle side view. The standard coordinate system is determined based on the installation perspective information corresponding to the vehicle side view.

9. A vehicle window positioning device, characterized in that: The device includes a memory, a processor, and a vehicle window positioning program stored in the memory and executable on the processor. When the vehicle window positioning program is executed by the processor, the steps of the vehicle window positioning method according to any one of claims 1 to 8 are implemented.

10. A storage medium, characterized in that: The storage medium stores a vehicle window positioning program, which, when executed by a processor, implements the steps of the vehicle window positioning method according to any one of claims 1 to 8.

Images