A method and device for generating a detection sample loading path

Abstract

The present invention discloses a method and device for generating a detection sample loading path, the method comprising: collecting real-time images in a target area, obtaining a panoramic view of a sample loading device performing a panoramic scan in the target area; performing visual analysis based on the collected real-time images, and determining the position and posture of the sample loading device under visual analysis; matching the panoramic view with pre-stored state space data, determining the current environmental characteristics and the position and posture of the sample loading device under panoramic scanning; correcting the position and posture of the sample loading device under visual analysis; generating a continuous positioning trajectory of the sample loading device, and determining the continuous positioning trajectory of the sample loading device as the detection sample loading path trajectory. The present invention generates a continuous and accurate detection sample loading path trajectory through multi-source data fusion and real-time correction, significantly improving the system's positioning accuracy, environmental perception capability, and reliability of the path trajectory, and ensuring efficient and accurate loading of detection samples in the warehouse.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a method and device for generating a detection sample loading path. Background Art

[0002] With the continuous development of modern logistics and warehousing management, the level of automation and intelligence in warehouses is constantly improving. As an important part of warehouse automation, automated guided vehicles (AGVs) are widely used in tasks such as material handling and sample loading. In order to ensure that AGVs can complete tasks efficiently and accurately, accurate positioning and path planning are essential. However, traditional positioning and path generation methods face many challenges in practical applications.

[0003] At present, traditional positioning technologies such as GPS are limited by signal coverage and accuracy issues in indoor environments and cannot meet the needs of high-precision positioning. Although visual positioning technology can provide high positioning accuracy, it is easily affected by occlusion and light changes in complex environments, resulting in unstable positioning. In addition, a single visual positioning or radar scanning method is difficult to fully perceive environmental characteristics and cannot achieve accurate path generation.

[0004] Existing multi-sensor fusion methods can improve positioning accuracy and environmental perception capabilities by combining multiple positioning methods. However, existing multi-sensor fusion methods mostly focus on simple data fusion and lack effective correction mechanisms, resulting in positioning errors and path discontinuities in the system in dynamic environments.

[0005] Therefore, there is an urgent need to improve the positioning accuracy and reliability of path generation of AGV in warehouse environments in order to improve the efficiency and accuracy of warehouse automation management. Summary of the invention

[0006] In view of the defects existing in the above-mentioned prior art, the present invention provides a method for generating a detection sample loading path, the method comprising:

[0007] Collect real-time images of the target area, and obtain a panoramic image of the sample loading device performing a panoramic scan in the target area;

[0008] Perform visual analysis based on the collected real-time images to determine the position and posture of the sample loading equipment under visual analysis;

[0009] Match the panoramic image with the pre-stored state space data to determine the current environmental characteristics and the position and posture of the sample loading equipment under the panoramic scan;

[0010] Based on the current environmental characteristics and the posture of the sample loading equipment under panoramic scanning, the posture of the sample loading equipment under visual analysis is corrected;

[0011] Based on the corrected position and posture, a continuous positioning trajectory of the sample loading device is generated, and the continuous positioning trajectory of the sample loading device is determined as the detection sample loading path trajectory.

[0012] The real-time image acquisition in the target area includes:

[0013] The sample to be tested is initially positioned to determine the grid area of ​​the sample to be tested within the target area; the image acquisition device corresponding to the grid area is called according to the determined grid area to obtain the real-time image within the grid area.

[0014] The method of calling an image acquisition device corresponding to the grid area according to the determined grid area to obtain a real-time image in the grid area includes:

[0015] According to the determined grid area, the grid coordinate results of the grid area are given;

[0016] Query the image acquisition device corresponding to the grid coordinates in the mapping table;

[0017] Sending a control command to a corresponding image acquisition device to acquire real-time image data;

[0018] After receiving the control command, the image acquisition device starts to capture the image data of the grid area in real time.

[0019] Among them, visual analysis is performed based on the collected real-time images to determine the current position of the sample loading device, including:

[0020] Acquire real-time image data in the first cycle and perform feature extraction;

[0021] The position of the sample loading device is determined by matching the pre-stored sample template with the features extracted from the real-time image data;

[0022] Determine the direction of the sample loading device based on the shape and characteristic point distribution of the sample loading device;

[0023] According to the direction of the sample loading device, the angle between the main axis direction and the reference direction is calculated to determine the orientation of the sample loading device;

[0024] The pixel coordinates of the sample loading device in the real-time image data are converted into actual coordinates, and the direction angle information of the sample loading device is output.

[0025] The panoramic image of the panoramic scan performed by the sample loading device in the target area is a panoramic image obtained after the sample loading device performs a panoramic scan of the target area by using a radar in the target area.

[0026] The process of obtaining a panoramic image of the sample loading device performing a panoramic scan in the target area includes:

[0027] The sample loading device performs a panoramic scan of the target area in the second cycle; each scan collects point cloud data of the environment in the target area and records the distance and angle information from the sample loading device to each characteristic point of the environment.

[0028] The second period is greater than the first period.

[0029] Among them, the panoramic image is matched with the pre-stored state space data to determine the current environmental characteristics and the position and posture of the sample loading equipment under the panoramic scan, including:

[0030] Extract environmental feature points from the panoramic image; match the feature points of the environment in the panoramic image with the feature points in the pre-stored state space data; screen and analyze the matching results, remove erroneous matching results, give the current environmental features, and calculate the rotation vector and translation vector of the sample loading device to determine the position and direction of the sample loading device.

[0031] Among them, the matching results are screened and analyzed to give the current environment characteristics, including:

[0032] The matching results are analyzed and processed to obtain the best transformation matrix, and the matching evaluation parameters of the best transformation matrix are given; wherein the matching evaluation parameters of the best transformation matrix include the number of supports, matching ratio, average error and standard deviation.

[0033] The matching results are analyzed and processed to obtain the best transformation matrix, including:

[0034] S31, randomly selecting a minimum sample set from all matching point pairs; wherein the matching point pairs are feature point pairs formed by matching feature points of the environment in the panoramic image with feature points in the pre-stored state space data;

[0035] S32, calculating the assumed rotation matrix and translation vector to form a transformation matrix;

[0036] S33, for each matching point pair, applying the transformation matrix to calculate the predicted point cloud coordinates of the point cloud coordinates in the matching point pair in the state space data;

[0037] S34, calculating the Euclidean distance between the predicted point cloud coordinates and the actual point cloud coordinates in the state space data;

[0038] S35, for each matching point pair, determining whether the Euclidean distance is less than a preset threshold value, if the Euclidean distance is less than the preset threshold value, it is considered that the matching point pair supports the current transformation matrix;

[0039] S36, repeat steps S31 to S35 until convergence, and give the transformation matrix with the largest number of supports as the optimal transformation matrix.

[0040] Among them, if the matching point pair supports the current transformation matrix, its support is 1, otherwise, the support is 0;

[0041] In step 36, steps S31 to S35 are repeated until convergence (a preset number of iterations), and the transformation matrix with the largest cumulative support is given as the optimal transformation matrix.

[0042] Among them, based on the current environmental characteristics and the posture of the sample loading equipment under panoramic scanning, the posture of the sample loading equipment under visual analysis is corrected, including:

[0043] It will support the number and matching ratio Multiply them together to get a preliminary matching quality score. Specifically: Preliminary Match Quality Score ;

[0044] For the average error and standard deviation of error Calculate the geometric mean to get the accuracy score , the accuracy score Specifically: Accuracy score ;

[0045] Finally, the preliminary matching quality score and accuracy score are nonlinearly superimposed to obtain the final modified weight index , the modified weight index The calculation method is: ;in, is a tuning parameter;

[0046] Afterwards, using the modified weight index Calculate the normalized correction factor ;

[0047] Based on the correction factor , as well as the position and direction of the sample loading device under panoramic scanning, the position and direction of the sample loading device under visual analysis are corrected to meet the following relationship:

[0048]

[0049] In the formula, The corrected position of the sample loading device, The corrected orientation of the sample loading device. Positioning of sample loading equipment under visual analysis, Direction for sample loading equipment under visual analysis, The position of the sample loading equipment under panoramic scanning, Direction for panoramic scanning of sample loading equipment.

[0050] Among them, the posture correction of the sample loading equipment under visual analysis includes:

[0051] After the panoramic scan is triggered, the posture of the sample loading equipment under visual analysis is corrected by using the current environmental characteristics and the posture of the sample loading equipment under panoramic scanning.

[0052] Wherein, based on the corrected position and posture, a continuous positioning trajectory of the sample loading device is generated, and the continuous positioning trajectory of the sample loading device is determined as the detection sample loading path trajectory, including:

[0053] Obtaining the pose of the sample loading device based on visual analysis only, and marking it as an uncorrected point, and obtaining the pose of the sample loading device based on visual analysis after correcting it, and marking it as a corrected point;

[0054] Assign weights to the corrected points and the uncorrected points, where the weight of the corrected points is 0.8 and the weight of the uncorrected points is 0.2;

[0055] Based on the weights of the corrected points and the uncorrected points, a smoothed trajectory point is obtained by smoothing each corrected point or uncorrected point.

[0056] Wherein, based on the weights of the corrected points and the uncorrected points, each corrected point or the uncorrected point is smoothed to obtain a smoothed trajectory point, including:

[0057] Perform weighted average processing on the coordinates of each corrected point or uncorrected point and the coordinates of its adjacent uncorrected point or corrected point to give a weighted average value of each corrected point or uncorrected point;

[0058] The weighted average of each corrected point or uncorrected point is used as the smoothed coordinate to obtain the smoothed trajectory point.

[0059] Among them, the weighted average is calculated to satisfy the following relationship:

[0060] For each corrected or uncorrected point , whose smoothed coordinates The calculation is as follows: ,in, It is The weight of each adjacent point (uncorrected point or corrected point) is K, and K is half of the window size. The choice of window size depends on the application scenario and the amount of data (the number of adjacent points).

[0061] The present invention also discloses a device for generating a detection sample loading path. The device adopts the above-mentioned method for generating a detection sample loading path, and comprises:

[0062] A data acquisition module, which is used to collect real-time images in the target area and obtain a panoramic image of the sample loading device performing a panoramic scan in the target area;

[0063] A visual analysis module, which is used to perform visual analysis based on the collected real-time images and determine the position and posture of the sample loading device under visual analysis;

[0064] A panoramic analysis module, which is used to match the panoramic image with the pre-stored state space data to determine the current environmental characteristics and the position and posture of the sample loading equipment under panoramic scanning;

[0065] A posture correction module, which is used to correct the posture of the sample loading device under visual analysis based on the current environmental characteristics and the posture of the sample loading device under panoramic scanning;

[0066] The trajectory generation module is used to generate a continuous positioning trajectory of the sample loading device based on the corrected posture, and determine the continuous positioning trajectory of the sample loading device as the detection sample loading path trajectory.

[0067] The present invention also discloses a system for generating a detection sample loading path, the system comprising a memory and a processor, the memory storing a computer program, and the processor being used to execute the computer program in the memory to implement the above method.

[0068] The present invention uses panoramic scanning to match pre-stored state space data to enhance environmental perception and ensure the accuracy of environmental feature recognition. In a dynamic environment, the movement trajectory of the sample loading equipment (AGV equipment) is easily affected by noise and errors, resulting in discontinuous or inaccurate path trajectories. The present invention generates a continuous and accurate positioning trajectory of the sample loading equipment by correcting the visual analysis results, ensuring the reliability of the sample loading path detection.

[0069] The present invention generates a continuous and accurate path trajectory for testing sample loading through multi-source data fusion and real-time correction, significantly improving the positioning accuracy, environmental perception capability and reliability of the path trajectory of the system, and ensuring efficient and accurate loading of testing samples in the warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0071] Figure 1 is a flow chart showing a method for generating a detection sample loading path according to an embodiment of the present invention;

[0072] Figure 2 is a flow chart showing a generation method of another embodiment;

[0073] Figure 3 is a schematic diagram showing a device for generating a detection sample loading path according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0076] It should be understood that although the terms first, second, third, etc. may be used to describe ... in the embodiments of the present invention, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of the present invention, the first ... may also be referred to as the second ..., and similarly, the second ... may also be referred to as the first ....

[0077] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0078] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0079] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0080] Traditional positioning methods are easily affected by factors such as occlusion and signal interference in complex indoor environments, resulting in insufficient positioning accuracy. The present invention combines UWB positioning, visual analysis and radar scanning, and comprehensively utilizes multiple positioning methods to improve positioning accuracy.

[0081] like Figure 1 As shown, the present invention discloses a method for generating a detection sample loading path, the method comprising:

[0082] Collect real-time images of the target area, and obtain a panoramic image of the sample loading device performing a panoramic scan in the target area;

[0083] Perform visual analysis based on the collected real-time images to determine the position and posture of the sample loading equipment under visual analysis;

[0084] Match the panoramic image with the pre-stored state space data to determine the current environmental characteristics and the position and posture of the sample loading equipment under the panoramic scan;

[0085] Based on the current environmental characteristics and the posture of the sample loading equipment under panoramic scanning, the posture of the sample loading equipment under visual analysis is corrected;

[0086] Based on the corrected position and posture, a continuous positioning trajectory of the sample loading device is generated, and the continuous positioning trajectory of the sample loading device is determined as the detection sample loading path trajectory. The sample loading device can be an AGV device.

[0087] In actual application scenarios, such as Figure 2 As indicated, the present invention discloses a method for generating a detection sample loading path, the method comprising:

[0088] Step 1: Use the UWB positioning method to perform initial positioning and determine the grid area of ​​the sample to be tested in the warehouse (target area);

[0089] Step 2, calling the warehouse camera corresponding to the grid area to obtain the real-time image of the grid area;

[0090] Step 3: Based on the real-time image acquired by the warehouse camera, the image processing algorithm is used to perform visual analysis to determine the current posture (including position and direction) of the AGV device;

[0091] Step 4: Use the radar on the AGV device to perform a panoramic scan to generate a panoramic image of the environment;

[0092] Step 5: Match the panoramic scanning result (panoramic image) with the pre-stored state space data to determine the current environmental features and the posture (including position and direction) of the AGV device under the panoramic scanning;

[0093] Step 6, based on the current environmental features and the posture of the AGV device under panoramic scanning, correct the posture of the AGV device obtained by the visual analysis;

[0094] Step 7: Based on the corrected position and posture of the AGV device, a continuous positioning trajectory of the AGV device is generated, and the continuous positioning trajectory of the AGV device is determined as the detection sample loading path trajectory.

[0095] This method combines UWB positioning, visual analysis and radar scanning to achieve high-precision sample positioning and continuous trajectory generation with real-time correction function. Its high-precision positioning and continuous trajectory generation make it suitable for scenarios that require precise positioning and path planning, such as power detection and warehouse management.

[0096] It can be seen that the present invention provides a method for generating a power detection sample loading path, which is applied in a detection warehouse for power materials to generate an accurate sample loading path.

[0097] First, the present invention uses UWB positioning technology for initial positioning to determine the grid area of ​​the sample to be tested in the warehouse. When the UWB positioning technology performs initial positioning, the corresponding UWB device may include a tag and a base station. Through the created grid system, each grid has a unique coordinate identifier. Next, the present invention calls the warehouse camera corresponding to the grid area to obtain a real-time image of the area. Based on the real-time image obtained by the camera, the image processing algorithm is used to perform visual analysis to determine the current position and direction of the AGV device.

[0098] Subsequently, a panoramic scan is performed using the radar on the AGV device. The radar generates a panoramic image of the environment through periodic scanning. The present invention matches the results of the panoramic scan with the pre-stored state space data to determine the current environmental characteristics, and thereby determines the relative distance between the current position and the reference point.

[0099] According to the determined current environmental features and the position and orientation of the AGV device under panoramic scanning, the position and orientation obtained by the previous visual analysis are corrected to ensure the accuracy of positioning.

[0100] Then, based on the corrected visual results, a continuous detection sample loading path trajectory is generated. The path generation module outputs the sample loading path trajectory by integrating the continuously corrected position information and direction data. The overall architecture of the present invention includes a positioning module (UWB positioning device and grid system), a visual module (camera and image processing unit), a scanning module (AGV device and radar scanning unit), a data processing module (state space database and data matching correction algorithm) and a path generation module (trajectory generation algorithm and data output interface).

[0101] Among them, the visual analysis service is triggered with a first cycle, and the radar scanning and repair positioning is triggered with a second cycle. The second cycle is greater than the first cycle. The radar scanning result corrects the positioning and direction obtained by the visual analysis service with the closest scanning time and video acquisition time.

[0102] In one embodiment, the UWB positioning system is started to ensure that all base stations and tags are working properly. The UWB tag on each AGV device (or sample to be tested) begins to periodically transmit ultra-wideband signals. Multiple UWB base stations in the warehouse simultaneously receive signals from the tags and record the arrival time. The UWB positioning system receives the timestamp data from the base station and uses triangulation to calculate the precise location of the tag. The tag location coordinates are mapped to the grid system to determine the grid area where it is currently located. The UWB positioning system outputs the unique coordinate identifier of the grid where the tag is located to complete the initial positioning.

[0103] In one embodiment, UWB positioning is used for initial positioning to determine the grid area of ​​the sample to be tested in the warehouse. UWB equipment configuration is required. A UWB tag is installed on each AGV device (or sample to be tested). These tags can emit ultra-wideband signals. Multiple UWB base stations are installed at fixed locations in the warehouse. These base stations are used to receive the signals sent by the tags and measure their arrival time.

[0104] The warehouse is divided into several grid areas, each grid has a unique coordinate identifier, and the grid position is represented by a two-dimensional coordinate system.

[0105] The coordinate identification of each grid is pre-stored in the system database to form a grid coordinate system.

[0106] UWB tags periodically transmit ultra-wideband signals, and multiple UWB base stations receive the signals from the tags and measure the time of arrival of the signals ToA.

[0107] The precise location of the UWB tag is calculated using triangulation based on the arrival times of the signals received from multiple base stations. Each base station records the timestamp of the signal arrival. Based on the known signal propagation speed (close to the speed of light), the distance between the tag and each base station is calculated. Using triangulation, the two-dimensional or three-dimensional position coordinates of the tag are solved based on the distance information of multiple base stations.

[0108] Grid area determination includes mapping the calculated label position coordinates into a pre-divided grid system.

[0109] Based on the label coordinates, the grid area to which it belongs is determined and identified by the unique coordinates of the output grid.

[0110] In one embodiment, the installation location of the camera (image acquisition device) is reasonably planned in the inspection warehouse of the power materials to ensure that each grid area is covered by at least one camera. Each camera has a unique identifier (ID) and is pre-associated with the coordinate identification of the grid area and stored in the system database. All cameras are connected to the central control system or server via a wired or wireless network. The camera is configured to transmit video streams or image data to the central control system in real time.

[0111] In one embodiment, a mapping table of grid coordinates and camera IDs is created in a system database, so that the corresponding camera can be quickly found and called according to the grid coordinates.

[0112] In one embodiment, the present invention receives the grid coordinate result from the UWB positioning module. The camera ID corresponding to the grid coordinate is queried in the mapping table. A control command is sent to the corresponding camera to obtain real-time image data. After the camera receives the control command, it starts to capture the image or video stream of the grid area in real time. The real-time image data is transmitted to the central control system through the network. The system receives the image data and stores it in the cache, and at the same time starts the image processing module for subsequent analysis.

[0113] In one embodiment, cameras are installed in each grid area in the warehouse to ensure that the camera covers all grid areas and assign a unique ID to each camera. The camera network is started to ensure that all cameras are connected to the central control system and can transmit image data in real time. The system receives the coordinate identifier (x, y) of the grid where the sample is located provided by the UWB positioning module. The system searches for the camera ID corresponding to the grid (x, y) in a pre-created mapping table, such as CamID_x_y. The system sends a control command to the camera CamID_x_y to obtain a real-time image. The camera CamID_x_y starts to capture the real-time image of the grid area. The camera transmits the real-time image to the central control system via the network. The central control system receives and caches the real-time image data, and then starts the image processing module.

[0114] Based on the real-time camera image data obtained from step 2, the image processing algorithm is used for visual analysis to determine the current position and direction of the AGV device. Image preprocessing is required, including grayscale and filtering, and then the Canny edge detection algorithm is used to extract edge information in the image.

[0115] After that, feature extraction is performed, and the shape features of the AGV device are identified through the shape contour detection algorithm Hough transform. The feature points of the AGV device are extracted using the SIFT scale-invariant feature transformation algorithm. The pre-stored AGV device template is matched with the features in the camera image to determine the location of the AGV device. The image segmentation algorithm GrabCut is used to separate the AGV device from the background to further improve the positioning accuracy.

[0116] Based on the shape and feature point distribution of the AGV device, the principal component analysis (PCA) algorithm is used to determine the direction of the AGV device. The angle between the main axis direction of the AGV device and the reference direction is calculated to determine the specific orientation of the AGV device.

[0117] Finally, the pixel coordinates of the AGV device in the image are converted into actual coordinates, combined with the calibration parameters of the camera (such as focal length and viewing angle); and the direction angle information of the AGV device is output.

[0118] In one embodiment, based on the shape and feature point distribution of the AGV device, the PCA algorithm is used to determine the direction of the AGV device, including:

[0119] The SIFT algorithm is used to extract key feature points of the AGV device in the image. The feature points include key positions such as edges and corners of the AGV device, and then principal component analysis (PCA) direction estimation is performed.

[0120] The extracted feature point coordinates are expressed as a matrix , where each row represents the coordinates of a feature point ,in:

[0121] : Feature point coordinate matrix.

[0122] : No. The horizontal coordinate of the feature point.

[0123] : No. The vertical coordinate of the feature point.

[0124] Decentralize and calculate the mean of feature point coordinates ;

[0125] in:

[0126] : The mean vector of feature point coordinates.

[0127] : The number of feature points.

[0128] : No. The coordinate vector of the feature points.

[0129] Decenter the feature point coordinate matrix: ;

[0130] in:

[0131] : The coordinate matrix of the decentralized feature points.

[0132] Calculate the covariance matrix of the decentralized feature point coordinates ;

[0133] in:

[0134] : Covariance matrix of feature point coordinates.

[0135] Then calculate the covariance matrix The eigenvalues ​​and eigenvectors of are: ;

[0136] in:

[0137] : Covariance matrix No. feature vectors.

[0138] : Covariance matrix No. A feature quantity.

[0139] Select the largest eigenvalue The corresponding eigenvector as the main axis direction.

[0140] : Maximum eigenvalue The corresponding eigenvector represents the main axis direction of the AGV device.

[0141] Through the feature vector Determine the main axis direction of the AGV equipment and define It is the main axis direction vector of the AGV device.

[0142] Set a reference direction (such as the positive x-axis direction of the image coordinate system) ).

[0143] definition is the reference direction vector.

[0144] Calculate the angle between the main axis direction of the AGV equipment and the reference direction ,

[0145] in,

[0146] : The angle between the main axis direction of the AGV equipment and the reference direction.

[0147] : The dot product of the principal axis direction vector and the reference direction vector.

[0148] : Magnitude of the principal axis direction vector.

[0149] : Magnitude of the reference direction vector.

[0150] Finally, the pixel coordinates of the AGV device in the image are converted to actual coordinates, including:

[0151] Determine the intrinsic matrix of camera calibration parameters , including focal length , principal point coordinates :

[0152] ,

[0153] in,

[0154] : The camera's intrinsic parameter matrix.

[0155] : Camera in Axis and The focal length along the axis.

[0156] : The coordinates of the camera's principal point (optical center) in the image.

[0157] Convert pixel coordinates to normalized coordinates. Suppose the pixel coordinates of the AGV device in the image are ,in is the pixel coordinate of the AGV device in the image.

[0158] Normalized coordinates, use the intrinsic matrix to convert pixel coordinates to normalized coordinates:

[0159] ,

[0160] in, : Normalized coordinate vector.

[0161] : The inverse matrix of the internal parameter matrix.

[0162] Use the camera's external parameters (rotation matrix and translation vector ) Convert the normalized coordinates to actual 3D coordinates:

[0163] ,

[0164] in:

[0165] : The rotation matrix of the camera.

[0166] : The camera's translation vector.

[0167] : The coordinates of the AGV device in the actual space.

[0168] : Normalized coordinate vector.

[0169] In one embodiment, a radar on an AGV device is used to perform a panoramic scan and pre-generate a panoramic view of the environment, including a laser radar (LIDAR). After configuring its scanning frequency, resolution, and scanning range, the radar performs a 360-degree scan at a fixed period to capture point cloud data of the environment. The data of multiple scans are aligned and fused to generate a complete panoramic view of the environment. The point cloud data is stored and filtered and converted into rectangular coordinate system data.

[0170] In a certain embodiment, the pre-stored state space data generally includes characteristic points of the environment, structural information, etc. These data can be generated by prior environmental scanning, map construction or simulation.

[0171] In one embodiment, during the stage of positioning and collecting to generate a panoramic image, the radar performs a 360-degree panoramic scan at a preset period (second period). Each scan collects point cloud data of the environment and records the distance and angle information of each measurement point (feature point of the environment). The point cloud data collected by the radar is stored in a memory. A filtering algorithm (such as a mean filter) is used to remove noise points.

[0172] Perform coordinate transformation on point cloud data and convert polar coordinate data into rectangular coordinate system data. Convert to rectangular coordinates The formula is: , .

[0173] In one embodiment, the state space data is preprocessed and the environmental feature points and structure information are stored. The environmental feature points are extracted from the scan results and described. A matching algorithm is used to match the feature points in the scan results with the feature points in the state space data. The RANSAC algorithm is used to filter the matching results and remove false matches. The rotation vector and translation vector of the device are calculated by the PnP algorithm to determine the position and posture of the device.

[0174] In one embodiment, the environmental feature point extraction includes extracting key feature points of the environment (such as corner points, edges, etc.) from the panoramic scanning results. The feature points are described using feature descriptors (such as SIFT feature descriptors), which can facilitate the subsequent matching of the panoramic image with the pre-stored state space data. During the matching, the feature matching algorithm FLANN is used to match the feature points in the panoramic scanning results with the feature points in the pre-stored state space data.

[0175] In a certain embodiment, feature matching may include calculating the Euclidean distance between feature points to be matched, and selecting two feature points with the smallest Euclidean distance for pairing.

[0176] In one embodiment, a random sampling consensus algorithm RANSAC is used to screen the matching results, remove false matches, and retain correct matches. The RANSAC algorithm performs matching, including: randomly selecting a set of matching point pairs. Calculating a hypothetical transformation matrix. Evaluating the support of the transformation matrix (i.e., how many point pairs conform to the transformation). Repeating the above process until the transformation matrix with the highest support is found. The advantage of RANSAC is that it can find reliable data parameters in the presence of a large amount of noisy data.

[0177] In one embodiment, the pose (position and orientation) of the AGV device is calculated by matching feature points, and the perspective n-point (PnP) algorithm is used to calculate the specific position and orientation of the AGV device in the environment based on the matching points. The PnP algorithm is used to estimate the pose (position and orientation) of the AGV device from 2D image points and corresponding 3D space points. The PnP algorithm is often used in fields such as computer vision and robot navigation for pose estimation and positioning. The PnP algorithm includes: inputting matching feature point pairs, calculating the rotation vector and translation vector of the device based on the matching point pairs, and finally using these vectors to determine the pose of the AGV device.

[0178] In one embodiment, suppose there is a set of matching point pairs ,in are the coordinates in the source point cloud (the point cloud coordinates of the feature points of the environment in the panorama), is the coordinate in the target point cloud (the feature point cloud coordinate in the state space data). The RANSAC algorithm is used to screen the matching results and finally evaluate the matching degree of the best transformation matrix.

[0179] In one embodiment, the RANSAC algorithm is used to screen the matching results, including:

[0180] Randomly select a minimum sample set (e.g., 4 or 5 pairs of points) from all matching point pairs.

[0181] Compute the assumed rotation matrix using least squares or other methods and translation vector , forming a transformation matrix .

[0182] For each matching point pair , apply the transformation matrix Calculate the coordinates of the source point cloud Predicted point cloud coordinates in the target point cloud .

[0183] Calculate the predicted point cloud coordinates and actual point cloud coordinates The Euclidean distance between: .

[0184] For each matching point pair, determine the distance Is it less than the preset threshold? Support indicator function = 1 indicates .

[0185] If the support indicator function is , then the point pair is considered to support the current transformation matrix.

[0186] Repeat the above process, that is, repeat the random selection and evaluation process until convergence (reaching the preset number of iterations), record the support each time and select the best transformation matrix. Specifically, select the transformation matrix with the highest support as the final transformation matrix.

[0187] In one embodiment, the optimal transformation matrix is ​​obtained based on the RANSAC algorithm. , and a threshold has been set , then determine the current environment characteristics, that is, determine the best transformation matrix Matching evaluation parameters.

[0188] In one embodiment, the optimal transformation matrix The matching evaluation parameters include the number of supports, matching ratio, average error and standard deviation, among which:

[0189] Number of support : This is the number of matching point pairs that meet the distance condition, which reflects the number of point pairs that meet the transformation relationship, that is, the number of matching point pairs that meet the optimal transformation matrix.

[0190] Matching ratio : This is the ratio of the number of supports to the total number of matching point pairs, reflecting the ratio of point pairs that satisfy the transformation relationship to the total number of point pairs, that is, the ratio of the number of matching point pairs that satisfy the optimal transformation matrix to the total number of matching point pairs.

[0191] Average Error : This is the average Euclidean distance of all matching point pairs that meet the distance conditions, reflecting the average matching error, that is, the average Euclidean distance of matching point pairs that meet the optimal transformation matrix.

[0192] Standard Deviation : This is the standard deviation of the distance of the matching point pairs that meet the distance condition, which reflects the distribution of the matching error, that is, the standard deviation of the Euclidean distance of the matching point pairs that meet the optimal transformation matrix.

[0193] In one embodiment, calculating the support quantity includes:

[0194] Initialize the counter to 0, that is .

[0195] Traverse all matching point pairs For each pair, apply the transformation matrix , and calculate the Euclidean distance between the predicted location and the actual location If this distance is less than the threshold ,Right now , then the counter increases by one, that is, Increase by 1.

[0196] The final value of this counter is the number of supports .

[0197] In one embodiment, the matching ratio is calculated ,include:

[0198] Total number of matching points is the total number of all matching point pairs.

[0199] The amount of support that will be obtained Divide by the total number of matching point pairs to get the matching ratio That is, calculate the matching ratio: .

[0200] In one embodiment, calculating the average error includes:

[0201] Initialize a variable and set it to 0. Also traverse all matching point pairs, for each matching point pair, if its Euclidean distance is less than the threshold , then add this distance to the initialized variable. Finally, divide this variable by the number of supports , we get the average error .

[0202] Specifically, calculate the average error ,include:

[0203] Initialization error sum . Traverse all matching point pairs : Calculate distance .if ,Will Add to total_error. Calculate the average error of the support point pairs: .

[0204] In one embodiment, calculating the standard deviation includes:

[0205] Initialize a variable again and set it to 0. Traverse all the matching point pairs, for each pair, if its Euclidean distance is less than the threshold , then add the square of the result of subtracting the average error from this distance to the initialized variable. Finally, divide this variable by the number of supports Then take the square root and you get the standard deviation. .

[0206] Specifically, calculate the standard deviation of the error ,include:

[0207] Initialize the error sum of squares: . Traverse all matching point pairs , calculate the distance .if ,calculate And add it to total_squared_error. Calculate the standard deviation of the error, using the calculated error sum total_squared_error and the number of support point pairs Calculate the standard deviation: .

[0208] In one embodiment, in step 6, the position and direction of the AGV device obtained by the visual analysis are corrected by first determining the following parameters:

[0209] AGV equipment position and orientation obtained by visual analysis:

[0210] AGV equipment location for visual analysis

[0211] The visual direction (orientation angle) of the AGV device

[0212] The position and direction of the AGV equipment obtained from the panoramic scanning results:

[0213] AGV equipment position determined for panoramic scanning

[0214] AGV device orientation (heading angle) determined for panoramic scanning

[0215] Matching parameters for panoramic scanning: : Number of support, : Matching ratio, Average error, : Standard deviation of the error.

[0216] Based on the current environmental features and the posture (position and direction) of the AGV device under panoramic scanning, the position and direction of the AGV device obtained by the visual analysis are corrected, including:

[0217] The number of support and matching ratio Multiply them together to get a preliminary matching quality score. Then, for the average error and standard deviation of error , and calculate their geometric mean.

[0218] Among them, the support number and matching ratio Multiply them together to get a preliminary matching quality score. Specifically: Preliminary Match Quality Score .

[0219] Among them, for the average error and standard deviation of error , calculate their geometric mean and obtain the accuracy score , the accuracy score Specifically: Accuracy score .

[0220] Finally, these two parts are nonlinearly superimposed to obtain the final modified weight index , the modified weight index The calculation method is: .in, is a tuning parameter that controls the balance between matching quality and accuracy.

[0221] Afterwards, using the modified weight index Calculate the normalized correction factor .

[0222] Based on the correction factor The position of the AGV device obtained by the visual analysis is corrected, and the corrected AGV device position The calculation formula is:

[0223] ,

[0224] Specifically expanded as follows:

[0225] ,

[0226] ,

[0227] .

[0228] Based on the correction factor The movement direction of the AGV device obtained by the visual analysis is corrected, and the corrected AGV device direction (orientation angle) The calculation formula is:

[0229] .

[0230] In one embodiment, the specific implementation steps of step 6 may be:

[0231] Step s61. Calculate the preliminary matching quality score: ;

[0232] Step s62. Calculate the accuracy score: ;

[0233] Step s63. Determine whether to adjust the parameters .

[0234] Step s64. Calculate the correction factor: ;

[0235] Step s65. Position correction: Correct the position using the correction factor: ;

[0236] Specifically expanded as follows:

[0237] ,

[0238] .

[0239] Step s66. Direction correction: Correct the direction using the correction factor: .

[0240] In one embodiment, the step 7 generates a continuous detection sample loading path trajectory, including:

[0241] Recording of initial position and direction: At the initial stage of generating the test sample loading path, record the initial position coordinates (x, y, z) and direction angle (theta) of the device. These initial data will be used as the starting point for generating the path trajectory and stored in a trajectory list.

[0242] Continuous data collection and updating:

[0243] The visual analysis service is triggered in a shorter cycle (the first cycle) to collect the location information and direction data of the AGV equipment. Each time the visual analysis service is triggered, the current device location and direction are obtained and recorded.

[0244] The radar scanning service is triggered with a longer period (second period) to correct the positioning of the device. Optionally, the second period is equal to N multiples of the first period, where N is a positive integer greater than 2.

[0245] After each radar scan is triggered, the radar data is used to correct the positioning and direction of the visual analysis. For the corrected visual positioning data, the corrected visual data replaces the corresponding pre-corrected visual data in the track list, and the corrected and uncorrected visual data are marked in the track list.

[0246] That is, in each cycle of the visual analysis service, the corrected position information and direction data are recorded, and each corrected data point is added to the trajectory list to ensure that each point in the trajectory list is based on the position information of the latest correction result, and then the trajectory is smoothed to obtain the detection sample loading path trajectory.

[0247] In one embodiment, the trajectory data is exported, and the smoothed trajectory data is exported as a file (such as a text file or a CSV file), and each line records the coordinates (x, y, z) and direction (theta) of a position point. A trajectory diagram is generated using a graphical tool to display the motion trajectory of the AGV device along the detection sample loading path.

[0248] Use graphical tools to generate trajectory diagrams to show the motion trajectory of the AGV equipment along the inspection sample loading path, providing an intuitive visual display.

[0249] In one embodiment, the corrected and uncorrected visual data are annotated in the track list and labeled as "corrected point" or "uncorrected point".

[0250] The corrected points are the visual analysis results corrected by the radar scanning results, while the uncorrected points are the direct visual analysis results.

[0251] Assign a higher weight to the corrected points, such as 0.8. Assign a lower weight to the uncorrected points, such as 0.2.

[0252] When smoothing each sample point, its own weight and the weights of neighboring points are taken into account.

[0253] The weighted average method is used to calculate the smoothed trajectory points.

[0254] In one embodiment, the trajectory list is traversed, and for each point in the trajectory list, its smoothed coordinates are calculated. The smoothed coordinates are the weighted average of its own coordinates and the coordinates of adjacent points.

[0255] Calculate the weighted average: For each point , whose smoothed coordinates The calculation is as follows: ,in, It is The weights of neighboring points, It is half of the window size. The choice of window size depends on the application scenario and the amount of data (the number of adjacent points).

[0256] Finally, the trajectory data after weighted smoothing is exported as a file, and a trajectory diagram is generated using a graphical tool to display the motion trajectory of the equipment along the sample loading path.

[0257] The present invention uses radar panoramic scanning to match pre-stored state space data, which enhances environmental perception and ensures the accuracy of environmental feature recognition. In a dynamic environment, the movement trajectory of the equipment is easily affected by noise and errors, resulting in discontinuous or inaccurate path trajectories. The present invention generates a continuous and accurate equipment positioning trajectory by performing radar scanning correction on the visual analysis results, ensuring the reliability of the loading path.

[0258] The present invention generates a continuous and accurate path trajectory for testing sample loading through multi-source data fusion and real-time correction, significantly improving the positioning accuracy, environmental perception capability and reliability of the path trajectory of the system, and ensuring efficient and accurate loading of testing samples in the warehouse.

[0259] like Figure 3 As shown, the present invention also discloses a device for generating a detection sample loading path, using the above-mentioned method for generating a detection sample loading path, the generating device comprises:

[0260] A data acquisition module is used to collect real-time images in the target area and obtain a panoramic image of the AGV device performing a panoramic scan in the target area;

[0261] A visual analysis module, which is used to perform visual analysis based on the collected real-time images and determine the position and posture of the AGV equipment under visual analysis;

[0262] A panoramic analysis module is used to match the panoramic image with the pre-stored state space data to determine the current environmental characteristics and the position and posture of the AGV device under panoramic scanning;

[0263] The posture correction module is used to correct the posture of the AGV device under visual analysis based on the current environmental characteristics and the posture of the AGV device under panoramic scanning;

[0264] The trajectory generation module is used to generate a continuous positioning trajectory of the AGV device based on the corrected posture, and determine the continuous positioning trajectory of the AGV device as the detection sample loading path trajectory.

[0265] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, but is not intended to limit the present invention. All modifications, substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A method for generating a detection sample loading path, characterized in that: The method comprises: Collect real-time images of the target area, and obtain a panoramic image of the sample loading device performing a panoramic scan in the target area; Perform visual analysis based on the collected real-time images to determine the position and posture of the sample loading equipment under visual analysis; Matching the panoramic image with the pre-stored state space data to determine the current environmental features and the position and orientation of the sample loading device under the panoramic scan, including: extracting environmental feature points from the panoramic image; matching the environmental feature points in the panoramic image with the feature points in the pre-stored state space data; screening and analyzing the matching results to give the current environmental features, and calculating the rotation vector and translation vector of the sample loading device to determine the position and orientation of the sample loading device; Based on the current environmental characteristics and the posture of the sample loading equipment under panoramic scanning, the posture of the sample loading equipment under visual analysis is corrected; Based on the corrected position and posture, a continuous positioning trajectory of the sample loading device is generated, and the continuous positioning trajectory of the sample loading device is determined as the detection sample loading path trajectory; Among them, the matching results are screened and analyzed to give the current environment characteristics, including: The matching results are analyzed and processed to obtain the best transformation matrix, and the matching evaluation parameters of the best transformation matrix are given; wherein the matching evaluation parameters of the best transformation matrix include the number of supports, matching ratio, average error and standard deviation.

2. A method for generating a detection sample loading path according to claim 1, characterized in that: Collect real-time images of the target area, including: The sample to be tested is initially positioned to determine the grid area of ​​the sample to be tested within the target area; the image acquisition device corresponding to the grid area is called according to the determined grid area to obtain the real-time image within the grid area.

3. A method for generating a detection sample loading path according to claim 2, characterized in that: Calling an image acquisition device corresponding to the grid area according to the determined grid area to obtain a real-time image in the grid area includes: According to the determined grid area, the grid coordinate results of the grid area are given; Query the image acquisition device corresponding to the grid coordinates in the mapping table; Sending a control command to a corresponding image acquisition device to acquire real-time image data; After receiving the control command, the image acquisition device starts to capture the image data of the grid area in real time.

4. A method for generating a detection sample loading path according to claim 1, characterized in that: Based on the collected real-time images, visual analysis is performed to determine the current position of the sample loading device, including: Acquire real-time image data in the first cycle and perform feature extraction; The position of the sample loading device is determined by matching the pre-stored sample template with the features extracted from the real-time image data; Determine the direction of the sample loading device based on the shape and characteristic point distribution of the sample loading device; According to the direction of the sample loading device, the angle between the main axis direction and the reference direction is calculated to determine the orientation of the sample loading device; The pixel coordinates of the sample loading device in the real-time image data are converted into actual coordinates, and the direction angle information of the sample loading device is output.

5. A method for generating a detection sample loading path according to claim 4, characterized in that: Obtain a panoramic image of the sample loading device performing a panoramic scan in the target area, including: The sample loading device performs a panoramic scan of the target area in the second cycle; each scan collects point cloud data of environmental feature points in the target area, and records the distance and angle information from the sample loading device to each environmental feature point.

6. A method for generating a detection sample loading path according to claim 1, characterized in that: Analyze and process the matching results to obtain the best transformation matrix, including: S31, randomly selecting a minimum sample set from all matching point pairs; wherein the matching point pairs are feature point pairs formed by matching feature points of the environment in the panoramic image with feature points in the pre-stored state space data; S32, calculating the assumed rotation matrix and translation vector to form a transformation matrix; S33, for each matching point pair, applying the transformation matrix to calculate the predicted point cloud coordinates of the point cloud coordinates in the matching point pair in the state space data; S34, calculating the Euclidean distance between the predicted point cloud coordinates and the actual point cloud coordinates in the state space data; S35, for each matching point pair, determining whether the Euclidean distance is less than a preset threshold value, if the Euclidean distance is less than the preset threshold value, it is considered that the matching point pair supports the current transformation matrix; S36, repeat steps S31 to S35 until convergence, and give the transformation matrix with the largest number of supports as the optimal transformation matrix.

7. A method for generating a detection sample loading path according to claim 1 or 6, characterized in that: Based on the current environmental characteristics and the posture of the sample loading equipment under panoramic scanning, the posture of the sample loading equipment under visual analysis is corrected, including: The number of support and matching ratio Multiply them together to get a preliminary matching quality score. Specifically: ； For the average error and standard deviation of error Calculate the geometric mean to get the accuracy score , the accuracy score Specifically: ； The initial matching quality score and accuracy score are nonlinearly superimposed to obtain the final modified weight index. The calculation method is: ;in, To adjust the parameters; Calculate the normalized correction factor using the correction weight index ; Based on the correction factor, and the position and orientation of the sample loading device under panoramic scanning, the position and orientation of the sample loading device under visual analysis are corrected.

8. A device for generating a detection sample loading path, characterized in that: The method for generating a detection sample loading path according to any one of claims 1 to 7 is adopted, and the generating device comprises: A data acquisition module, which is used to collect real-time images in the target area and obtain a panoramic image of the sample loading device performing a panoramic scan in the target area; A visual analysis module, which is used to perform visual analysis based on the collected real-time images and determine the position and posture of the sample loading device under visual analysis; A panoramic analysis module is used to match the panoramic image with the pre-stored state space data to determine the current environmental features and the position and orientation of the sample loading device under panoramic scanning, including: extracting environmental feature points from the panoramic image; matching the environmental feature points in the panoramic image with the feature points in the pre-stored state space data; screening and analyzing the matching results to give the current environmental features, and calculating the rotation vector and translation vector of the sample loading device to determine the position and orientation of the sample loading device; A posture correction module, which is used to correct the posture of the sample loading device under visual analysis based on the current environmental characteristics and the posture of the sample loading device under panoramic scanning; A trajectory generation module, which is used to generate a continuous positioning trajectory of the sample loading device based on the corrected position and posture, and determine the continuous positioning trajectory of the sample loading device as the detection sample loading path trajectory; Among them, the matching results are screened and analyzed to give the current environment characteristics, including: The matching results are analyzed and processed to obtain the best transformation matrix, and the matching evaluation parameters of the best transformation matrix are given; wherein the matching evaluation parameters of the best transformation matrix include the number of supports, matching ratio, average error and standard deviation.

Images