Safety inspection method and device based on point cloud processing and robot

Abstract

The application is suitable for the technical field of point cloud data processing, and provides a safety inspection method and device based on point cloud processing and a robot, the method comprising: acquiring true color image data and depth image data; detecting whether a license plate exists in the true color image data; in the case that the true color image data contains a license plate, acquiring a detection frame of the license plate; performing conversion processing on the pixel points of the depth image data in the region corresponding to the detection frame of the license plate to obtain point cloud data of the license plate; calculating the normal vector of the point cloud data; calculating the distance between the robot and the central axis of the license plate; judging the orientation of the robot relative to the license plate; calculating the included angle between the robot and the central axis of the license plate; moving to the central axis of the license plate; moving along the central axis of the license plate through the bottom of the vehicle, and collecting a vehicle bottom image when passing through the bottom of the vehicle to perform safety inspection through the vehicle bottom image.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of point cloud data processing, and particularly relates to a safety inspection method and device based on point cloud processing and a robot. BACKGROUND

[0002] With the development of economy, the number of cars is increasing, and the height and structure of cars make it difficult to check the bottom of the car, so that criminals hide dangerous goods in the bottom of the car. In order to protect safety and prevent dangerous goods from entering the country, the bottom of the car needs to be checked at important checkpoints such as ports and customs.

[0003] Currently, important checkpoints such as ports and customs still mainly rely on manual eye inspection, which is low in efficiency and difficult to ensure the safety of security personnel. SUMMARY

[0004] Therefore, the embodiments of the present application provide a safety inspection method and device based on point cloud processing, a robot and a computer readable storage medium, to solve the technical problem that the traditional technology mainly relies on manual eye inspection when checking the bottom of the car, which is low in efficiency and difficult to ensure the safety of security personnel.

[0005] The first aspect of the embodiments of the present application provides a safety inspection method based on point cloud processing, applied to a robot, comprising:

[0006] A first acquisition step: acquiring true color image data and depth image data;

[0007] A detection step: detecting whether a license plate exists in the true color image data;

[0008] A second acquisition step: acquiring a detection frame of the license plate in the case that the true color image data contains a license plate;

[0009] A conversion step: converting the pixel points of the depth image data in the region corresponding to the detection frame of the license plate to obtain point cloud data of the license plate;

[0010] A first calculation step: calculating the normal vector of the point cloud data;

[0011] A second calculation step: calculating the distance between the robot and the central axis of the license plate;

[0012] A judgment step: judging the orientation of the robot relative to the license plate;

[0013] A third calculation step: calculating the included angle between the robot and the central axis of the license plate;

[0014] Moving step: moving to the center axis of the license plate according to the distance between the robot and the center axis of the license plate, the orientation of the robot relative to the license plate, and the included angle between the robot and the center axis of the license plate;

[0015] Collecting step: moving along the center axis of the license plate through the bottom of the vehicle, collecting a vehicle bottom image when passing through the bottom of the vehicle, and performing safety inspection through the vehicle bottom image.

[0016] The second aspect of the embodiment of the application provides a safety inspection device based on point cloud processing, applied to a robot, comprising:

[0017] The first acquisition module is configured to acquire true color image data and depth image data.

[0018] The detection module is configured to detect whether a license plate exists in the true color image data.

[0019] The second acquisition module is configured to acquire a detection frame of the license plate in the case where the license plate exists in the true color image data.

[0020] The conversion module is configured to perform conversion processing on pixel points of the depth image data in a region corresponding to the detection frame of the license plate, to obtain point cloud data of the license plate.

[0021] The first calculation module is configured to calculate a normal vector of the point cloud data.

[0022] The second calculation module is configured to calculate a distance between the robot and the center axis of the license plate.

[0023] The judgment module is configured to judge an orientation of the robot relative to the license plate.

[0024] The third calculation module is configured to calculate an included angle between the robot and the center axis of the license plate.

[0025] The moving module is configured to move to the center axis of the license plate according to the distance between the robot and the center axis of the license plate, the orientation of the robot relative to the license plate, and the included angle between the robot and the center axis of the license plate.

[0026] The collecting module is configured to move along the center axis of the license plate through the bottom of the vehicle, collect a vehicle bottom image when passing through the bottom of the vehicle, and perform safety inspection through the vehicle bottom image.

[0027] The third aspect of the embodiment of the application provides a robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the safety inspection method based on point cloud processing in the first aspect when executing the computer program.

[0028] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the safety inspection method based on point cloud processing in the first aspect.

[0029] Compared with the prior art, the embodiment of the present application has the beneficial effects that: the present application controls the robot accurately according to the point cloud data of the license plate by recognizing the license plate, controls the robot to move along the central axis of the license plate through the bottom of the vehicle, and collects the vehicle bottom image when passing through the bottom of the vehicle, provides more basis for the vehicle bottom safety inspection, assists the port and customs to conduct the vehicle bottom safety inspection, improves the inspection efficiency, and guarantees the personal safety of the security personnel. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or related description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0031] Figure 1 A flowchart of a safety inspection method based on point cloud processing provided by the present application is shown;

[0032] Figure 2 A structural diagram of a safety inspection device based on point cloud processing provided by the present application is shown;

[0033] Figure 3 A structural diagram of a robot provided by the present application is shown. DETAILED DESCRIPTION

[0034] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail in order to avoid unnecessarily obscuring the present application.

[0035] Firstly, the present application provides a safety inspection method based on point cloud processing, which is applied to a robot.

[0036] In a possible implementation, the robot is provided with a camera, a line array camera and a fill light. The line array camera and the fill light are installed on the top of the robot, and are used to collect images of the vehicle bottom after the robot enters the vehicle bottom, to provide more basis for the vehicle bottom safety inspection, and to assist the port and customs in the vehicle bottom safety inspection.

[0037] Please refer to Figure 1 , Figure 1 A flowchart of a safety inspection method based on point cloud processing is shown. As Figure 1 shown, the safety inspection method based on point cloud processing can include the following steps:

[0038] The first acquisition step: acquiring true color image data and depth image data.

[0039] The true color image data, also known as RGB image data, can be acquired by a normal camera. The depth image data can be acquired by a depth camera, which uses special techniques (such as structured light, time of flight or binocular vision) to measure the distance of objects in the scene to the sensor and generate corresponding depth images.

[0040] The true color image data and the depth image data can be acquired simultaneously by the same camera, or can be acquired by different cameras.

[0041] In a possible implementation, after the first acquisition step, it further includes:

[0042] The alignment step: aligning the true color image data and the depth image data.

[0043] If the true color image data and the depth image data are acquired simultaneously by the same camera, the intrinsic alignment method can be used to align the true color image data and the depth image data. Specifically, the true color image and the depth image are aligned in the camera coordinate system using intrinsic parameters, including the focal length, optical center (image center), distortion parameters and the like of the camera.

[0044] If the true color image data and the depth image data are acquired by different cameras, the true color image and the depth image can be aligned in the world coordinate system. First, the pixel points in the depth image are converted into three-dimensional points in the world coordinate system using the depth value and the extrinsic information. Then, the pixel coordinates in the true color image and the extrinsic information of the camera are used to convert the points in the true color image into three-dimensional points in the world coordinate system. Finally, according to the coordinate correspondence, the true color image and the depth image can be aligned.

[0045] By aligning the true-color image data and the depth image data, it can be ensured that the license plate region determined in the true-color image data can be accurately mapped to the depth image data.

[0046] A detection step: detecting whether there is a license plate in the true-color image data.

[0047] Optionally, an open-source computer vision library (such as OpenCV) or a deep learning framework (such as TensorFlow, PyTorch) can be used to implement the license plate detection algorithm.

[0048] Specifically, first, image processing and computer vision techniques are used to extract features in the image to assist in license plate detection. Some commonly used features include color information, shape features, and edge information. According to the feature information, image segmentation techniques or object detection methods are used to extract image regions that may contain license plates, which usually have color features, shape features, or edge features of license plates. In the extracted regions, candidate license plate frames are generated, and edge detection algorithms or connected region analysis techniques can be used to find possible license plate boundaries. The generated candidate license plate frames are verified by feature analysis, pattern matching, machine learning, or deep learning methods to determine whether they actually contain license plates.

[0049] Further, if there is no license plate in the true-color image data, the process ends.

[0050] A second acquisition step: if there is a license plate in the true-color image data, acquire the detection frame of the license plate.

[0051] Specifically, the detection frame of the license plate can be acquired by a deep learning-based object detection algorithm (such as Faster R-CNN, YOLO, SSD, etc.).

[0052] A conversion step: converting the pixel points of the depth image data in the region corresponding to the detection frame of the license plate to obtain the point cloud data of the license plate.

[0053] Optionally, the pixel points of the depth image data in the license plate region are converted to point cloud data. The depth value of each pixel point can be represented as the Z coordinate in the point cloud, and the horizontal and vertical coordinates of the pixel point can be converted to the X and Y coordinates in the point cloud through the pixel coordinates of the depth image.

[0054] In one possible implementation, the conversion step includes:

[0055] A conversion sub-step: traversing the pixel points of the depth image data in the region corresponding to the detection frame of the license plate, and converting the pixel points of the depth image data according to Formula 1 to obtain the point cloud data of the license plate:

[0056]

[0057] wherein cx, cy, fx, fy represent internal parameters of the true color image data, depth represents the depth image data, and i, j represent column and row coordinates of a pixel point, respectively.

[0058] A statistical sub-step: counting the number of points in the point cloud data, and ending the process if the number of points is less than a preset number.

[0059] It should be noted that by counting the number of points in the point cloud data and ending the process when the number of points is less than a preset number, the deficiencies of the point cloud data can be identified in advance, unnecessary calculation and analysis processes can be avoided, which helps to ensure the quality and accuracy of the point cloud data and improve the overall processing efficiency.

[0060] A first calculation step: calculating the normal vector of the point cloud data.

[0061] The normal vector of the point cloud data is the basis for subsequent calculations of the distance of the robot from the central axis of the license plate, the orientation of the robot relative to the license plate, and the angle between the robot and the central axis of the license plate.

[0062] In one possible implementation, the first calculation step includes:

[0063] A first calculation sub-step: calculating the normal vector (α, β, γ) of the point cloud data by formula 2 T :

[0064]

[0065] wherein pointCloud represents a point cloud data set, C represents a covariance matrix, P center represents a license plate center point, n represents the number of point cloud data, P i represents the three-dimensional point information of the i-th point cloud data, and λ is the smallest eigenvalue.

[0066] In the present application, the license plate center point is taken as the midpoint in the width direction of the vehicle body.

[0067] It should be noted that the coordinates of the license plate center point can be calculated by calculating the coordinates of the top-left corner and the bottom-right corner of the detection frame of the license plate.

[0068] Further, the calculated normal vector can be smoothed to improve its accuracy and continuity. Common smoothing methods include weighted averaging or least squares fitting, etc.

[0069] A second computing step: computing the distance of the robot from the mid-axis of the license plate.

[0070] In a possible implementation, the second computing step comprises:

[0071] A second computing sub-step: computing the distance d of the robot from the mid-axis of the license plate by formula 3:

[0072]

[0073] wherein, represents the z-axis coordinate value of the license plate center point P center , represents the x-axis coordinate value of the license plate center point P center .

[0074] A judging step: judging the orientation of the robot relative to the license plate.

[0075] In a possible implementation, the judging step comprises:

[0076] A third computing sub-step: computing the intersection x0 of the mid-axis of the license plate and the x-axis of the world coordinate axis by formula 4:

[0077]

[0078] wherein, represents the z-axis coordinate value of the license plate center point P center , represents the x-axis coordinate value of the license plate center point P center .

[0079] A judging sub-step: in the case of intersection x0>0, judging that the robot is located on the right side of the license plate; in the case of intersection x0<0, judging that the robot is located on the left side of the license plate.

[0080] A third computing step: computing the included angle of the robot and the mid-axis of the license plate.

[0081] In a possible implementation, the third computing step comprises:

[0082] A fourth computing sub-step: computing the included angle ang of the robot and the mid-axis of the license plate by formula 5:

[0083] ang=arctan(αγ) formula 5.

[0084] Moving step: according to the distance d of the robot from the center axis of the license plate, the position of the robot relative to the license plate, and the included angle ang of the robot and the center axis of the license plate, moving to the center axis of the license plate.

[0085] In actual application, if the robot is on the right side of the center axis of the license plate, first rotate π / 2-ang degrees to the right, then move straight forward by a distance d, and finally rotate π / 2 degrees to the left; if x0 is less than zero, first rotate π / 2-ang degrees to the left, then move straight forward by a distance d, and finally rotate π / 2 degrees to the right.

[0086] In actual application, due to the influence of environmental factors such as light intensity, whether the road is slippery, etc., one-time movement to the center axis of the license plate is an ideal state, and fine adjustment will be performed in the actual processing process to increase the practicability of the algorithm.

[0087] Collecting step: passing through the bottom of the vehicle along the center axis of the license plate, collecting a vehicle bottom image when passing through the bottom of the vehicle, and performing safety inspection through the vehicle bottom image.

[0088] Optionally, the vehicle bottom image is collected by a linear array camera installed on the top of the robot. Further, a fill light can also be arranged on the top of the robot for use in dark environments. The vehicle bottom image data is recorded when passing through the bottom of the vehicle, the movement is stopped when driving out of the bottom of the vehicle, the vehicle bottom image is generated, and the vehicle bottom image is displayed on the terminal, so that the security officer can determine whether there is contraband by the vehicle bottom image.

[0089] Compared with the prior art, the embodiment of the present application has the beneficial effects that: the present application controls the robot by recognizing the license plate and using the point cloud data of the license plate, controls the robot to move through the bottom of the vehicle along the center axis of the license plate, collects a vehicle bottom image when passing through the bottom of the vehicle, provides more basis for vehicle bottom safety inspection, assists the port and customs in vehicle bottom safety inspection, improves the inspection efficiency, and ensures the personal safety of the security officer.

[0090] As Figure 2 The present application provides a safety inspection device 10 based on point cloud processing, applied to a robot.

[0091] Please refer to Figure 2 , Figure 2 The structure diagram of the safety inspection device based on point cloud processing is shown in the present application, as Figure 2 shown, a safety inspection device 20 based on point cloud processing includes:

[0092] The first acquisition module 201 is used for acquiring true color image data and depth image data.

[0093] The detection module 202 is configured to detect whether a license plate exists in the true-color image data.

[0094] The second acquisition module 203 is configured to acquire a detection frame of the license plate when the license plate exists in the true-color image data.

[0095] The conversion module 204 is configured to perform conversion processing on a pixel point of depth image data of a region corresponding to the detection frame of the license plate, to obtain point cloud data of the license plate.

[0096] The first calculation module 205 is configured to calculate a normal vector of the point cloud data.

[0097] The second calculation module 206 is configured to calculate a distance between the robot and a central axis of the license plate.

[0098] The judgment module 207 is configured to judge an orientation of the robot relative to the license plate.

[0099] The third calculation module 208 is configured to calculate an included angle between the robot and the central axis of the license plate.

[0100] The moving module 209 is configured to move to the central axis of the license plate according to the distance between the robot and the central axis of the license plate, the orientation of the robot relative to the license plate, and the included angle between the robot and the central axis of the license plate.

[0101] The collection module 210 is configured to move along the central axis of the license plate through a bottom of a vehicle, and collect a vehicle bottom image when passing through the bottom of the vehicle, to perform safety inspection through the vehicle bottom image.

[0102] In a possible implementation, the safety inspection device 20 based on point cloud processing further includes:

[0103] The alignment module is configured to align the true-color image data and the depth image data.

[0104] In a possible implementation, the conversion module 204 includes:

[0105] The conversion submodule is configured to traverse the pixel point of the depth image data of the region corresponding to the detection frame of the license plate, and perform conversion processing on the pixel point of the depth image data according to Formula 1, to obtain the point cloud data of the license plate.

[0106]

[0107] wherein, cx, cy, fx, fy represent internal parameters of the true-color image data, depth represents the depth image data, and i and j represent column coordinates and row coordinates of the pixel point, respectively.

[0108] a counting submodule, configured to count a number of points in the point cloud data, and end the process when the number of points is less than a preset number.

[0109] In a possible implementation, the first calculation module 205 includes:

[0110] a first calculation submodule, configured to calculate the normal vector (α, β, γ) of the point cloud data according to Formula 2. T :

[0111]

[0112] wherein pointCloud represents a point cloud data set, C represents a covariance matrix, P center represents a license plate center point, n represents a number of point cloud data, P i represents three-dimensional point information of the i th point cloud data, and λ is a minimum eigenvalue.

[0113] In a possible implementation, the second calculation module 206 includes:

[0114] a second calculation submodule, configured to calculate the distance d of the robot from the central axis of the license plate according to Formula 3.

[0115]

[0116] wherein, represents a z-axis coordinate value of the license plate center point P center , represents an x-axis coordinate value of the license plate center point P center .

[0117] In a possible implementation, the judging module 207 includes:

[0118] a third calculation submodule, configured to calculate the intersection x0 of the central axis of the license plate and the x-axis of the world coordinate axis according to Formula 4.

[0119]

[0120] wherein, represents a z-axis coordinate value of the license plate center point P center , represents an x-axis coordinate value of the license plate center point P center .

[0121] a judging submodule, configured to judge that the robot is located on the right side of the license plate when the intersection x0>0, and judge that the robot is located on the left side of the license plate when the intersection x0<0.

[0122] In a possible implementation, the third calculation module 208 comprises:

[0123] The fourth calculation sub-module is configured to calculate the included angle ang between the robot and the central axis of the license plate according to formula 5:

[0124] ang = arctan (αγ) formula 5.

[0125] The safety inspection device 20 based on point cloud processing provided by the application can realize the processes in the above method embodiments, and thus will not be described here again.

[0126] The virtual device provided by the application can be a terminal, a component in a terminal, an integrated circuit, or a chip.

[0127] Compared with the prior art, the embodiment of the application has the beneficial effect that the robot is accurately controlled according to the point cloud data of the license plate, the robot is controlled to move along the central axis of the license plate through the bottom of the vehicle, and the vehicle bottom image is collected when the robot passes through the bottom of the vehicle, thereby providing more basis for vehicle bottom safety inspection, assisting the port and customs in vehicle bottom safety inspection, improving the inspection efficiency, and ensuring the personal safety of the security personnel.

[0128] Figure 3 is a schematic diagram of a robot provided by an embodiment of the application. As shown in Figure 3 The robot 30 of this embodiment comprises a processor 300, a memory 301, and a computer program 302 stored in the memory 301 and executable on the processor 300, such as a safety inspection method based on point cloud processing program. The processor 300 implements the steps in each of the above safety inspection method embodiments based on point cloud processing when executing the computer program 302. Alternatively, the processor 300 implements the functions of each unit in each of the above device embodiments when executing the computer program 302.

[0129] For example, the computer program 302 can be divided into one or more units stored in the memory 301 and executed by the processor 300 to complete the application. The one or more units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 302 in the robot 30. For example, the computer program 302 can be divided into the specific functions of each module as follows:

[0130] The first acquisition module is configured to acquire true color image data and depth image data.

[0131] The detection module is configured to detect whether a license plate exists in the true color image data.

[0132] The second acquisition module is configured to acquire a detection frame of the license plate when the license plate exists in the true-color image data.

[0133] The conversion module is configured to perform conversion processing on a pixel point of the depth image data of a corresponding region of the detection frame of the license plate to obtain point cloud data of the license plate.

[0134] The first calculation module is configured to calculate a normal vector of the point cloud data.

[0135] The second calculation module is configured to calculate a distance between the robot and a center axis of the license plate.

[0136] The judgment module is configured to judge an orientation of the robot relative to the license plate.

[0137] The third calculation module is configured to calculate an included angle between the robot and the center axis of the license plate.

[0138] The moving module is configured to move to the center axis of the license plate according to the distance between the robot and the center axis of the license plate, the orientation of the robot relative to the license plate, and the included angle between the robot and the center axis of the license plate.

[0139] The acquisition module is configured to move along the center axis of the license plate through a bottom of a vehicle, and acquire a vehicle bottom image when passing through the bottom of the vehicle to perform safety inspection through the vehicle bottom image.

[0140] The robot can include but is not limited to a processor 300 and a memory 301. Those skilled in the art can understand that, Figure 3 is merely an example of a robot 30 and does not constitute a limitation on the robot 30, and can include more or fewer components than those shown, or combine certain components, or different components, for example, the robot can also include an input / output device, a network access device, a bus, etc.

[0141] The processor 300 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0142] The storage 301 can be an internal storage unit of the robot 30, such as a hard disk or a memory of the robot 30. The storage 301 can also be an external storage device of the robot 30, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the robot 30. Further, the storage 301 can include both the internal storage unit and the external storage device of the robot 30. The storage 301 is used to store the computer program and other programs and data required by the roaming control device. The storage 301 can also be used to temporarily store data that has been output or will be output.

[0143] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0144] It should be noted that the information interaction, execution process and the like between the above devices / units, since the same concept as the method embodiments of the present application, the specific functions and the technical effects brought about, specific can refer to the method embodiments part, here will not be repeated.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit, module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units, modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the above method embodiments, which will not be repeated here.

[0146] The embodiment of the present application also provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by a processor to realize the steps in each method embodiment.

[0147] The embodiment of the present application provides a computer program product, when the computer program product is run on a mobile terminal, the mobile terminal is caused to execute the steps in the above-mentioned various method embodiments.

[0148] The integrated unit, if in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the present application can implement all or part of the processes in the above-mentioned embodiment methods, and can be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps in the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer-readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / robot, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunications signal.

[0149] In the above-mentioned embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0150] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present text can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0151] In the embodiments of the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other manners. For example, the described apparatus / network device embodiment is merely illustrative, and the division of the modules or units can be different from the embodiment. For example, the plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0152] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, and can be located in one place or distributed on a plurality of network units.

[0153] It should be understood that, when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0154] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0155] As used in the specification and the appended claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to monitoring" depending on the context. Similarly, the phrases "if it is determined" or "if it is monitored that [the described condition or event] can be interpreted to mean "upon determining" or "in response to determining" or "upon monitoring [the described condition or event]" or "in response to monitoring [the described condition or event]" depending on the context.

[0156] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", etc. are merely used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0157] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in a various embodiment" or "in some embodiment" or "in other embodiments" in various places throughout this specification are not necessarily all referring to the same embodiment, but can refer to one or more of the same or different embodiments. Furthermore, the term "comprising" or "containing" or "including" or "having" or "characterized by" as used herein encompasses the presence of stated features, steps or components or integers, but does not preclude the presence or addition of one or more other features, steps, components or integers. The term "consisting essentially of" or "consisting of" as used herein exclude the presence of other features, steps, components or integers.

[0158] The above-described embodiments are merely intended to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A security inspection method based on point cloud processing, characterized in that, The application is applied to a robot, comprising: A first acquisition step: acquiring true color image data and depth image data; A detection step: detecting whether a license plate exists in the true color image data; A second acquisition step: acquiring a detection frame of the license plate in the case that the true color image data has the license plate; A conversion step: performing conversion processing on a pixel point of the depth image data in a region corresponding to the detection frame of the license plate to obtain point cloud data of the license plate; A first calculation step: calculating a normal vector of the point cloud data; A second calculation step: calculating a distance of the robot from a central axis of the license plate; A judgment step: judging an orientation of the robot relative to the license plate; A third calculation step: calculating an included angle of the robot and the central axis of the license plate; A moving step: moving to the central axis of the license plate according to the distance of the robot from the central axis of the license plate, the orientation of the robot relative to the license plate, and the included angle of the robot and the central axis of the license plate; A collection step: moving along the central axis of the license plate through a vehicle bottom, and collecting a vehicle bottom image when passing through the vehicle bottom to perform safety inspection through the vehicle bottom image, The conversion step comprises: A conversion sub-step: traversing the pixel point of the depth image data in the region corresponding to the detection frame of the license plate, and performing conversion processing on the pixel point of the depth image data according to Formula 1 to obtain the point cloud data of the license plate: Formula 1, wherein, cx , cy , fx , fy denote internal parameters of the true color image data, depth denote the depth image data, i , j denote the column and row coordinates of the pixel point, respectively. A statistical sub-step: counting a point quantity in the point cloud data, and ending the process in the case that the point quantity is less than a preset quantity.

2. The security inspection method based on point cloud processing according to claim 1, wherein, After the first acquisition step, further comprising: An alignment step: aligning the true color image data and the depth image data.

3. The security inspection method based on point cloud processing according to claim 1, wherein, The first calculation step comprises: A first calculation sub-step: calculating the normal vector of the point cloud data by formula 2 α , β , γ ) T : Formula 2, wherein pointCloud represents a point cloud data set, C denotes a covariance matrix, P center denotes a license plate center point, n denotes the number of point cloud data, P i denotes the third point information of the i point cloud data, λ is the minimum eigenvalue.

4. The security inspection method based on point cloud processing according to claim 3, characterized in that, The second calculation step comprises: Second calculation sub-step: calculating the distance of the robot from the mid-axis of the license plate by formula 3 d : Formula 3, wherein, represents the center point of the license plate P center of z the axis coordinate value, represents the center point of the license plate P center of x the axis coordinate value.

5. The security inspection method based on point cloud processing according to claim 3, wherein, The judgment step comprises: The third calculation sub-step: Calculate the relationship between the central axis of the license plate and the world coordinate axis using Formula 4. x Intersection of axes x 0: Formula 4, wherein, represents the center point of the license plate P center of z the axis coordinate value, represents the center point of the license plate P center of x the axis coordinate value; Judging sub-step: judging that the robot is on the right side of the license plate in case of intersection x 0>0; judging that the robot is on the left side of the license plate in case of intersection x 0<0.

6. The security inspection method based on point cloud processing according to claim 3, wherein, The third calculation step comprises: A fourth calculation sub-step: calculating the included angle ang of the robot and the central axis of the license plate through Formula 5: Formula 5.

7. A security inspection apparatus based on point cloud processing, characterized by, The safety inspection method based on point cloud processing comprises: A first acquisition module: used for acquiring true color image data and depth image data; A detection module: used for detecting whether a license plate exists in the true color image data; A second acquisition module: used for acquiring a detection frame of the license plate in the case that the true color image data has the license plate; A conversion module: used for performing conversion processing on a pixel point of the depth image data in a region corresponding to the detection frame of the license plate to obtain point cloud data of the license plate; A first calculation module: used for calculating a normal vector of the point cloud data; A second calculation module: used for calculating a distance of the robot from a central axis of the license plate; A judgment module: used for judging an orientation of the robot relative to the license plate; A third calculation module: used for calculating an included angle of the robot and the central axis of the license plate; A moving module: used for moving to the central axis of the license plate according to the distance of the robot from the central axis of the license plate, the orientation of the robot relative to the license plate, and the included angle of the robot and the central axis of the license plate; A collecting module is configured to move along a middle axis of the license plate through the bottom of the vehicle and collect a bottom image of the vehicle when passing through the bottom of the vehicle, so as to perform a security check through the bottom image.

8. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the security check method based on point cloud processing according to any one of claims 1-6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the security check method based on point cloud processing according to any one of claims 1-6.

Images