Vehicle clamp offset detection method and server

Abstract

The invention relates to the technical field of vehicles, in particular to a vehicle clamp offset detection method and a server, and is used for solving the problem of how to improve the positioning precision of a vehicle clamp. The method comprises the following steps: acquiring a to-be-identified image of a vehicle fixed on a vehicle clamp of a flexible drive conveying system in a preset area, and configuration information of a standard angular point corresponding to the preset area; performing image recognition on the to-be-recognized image, and determining at least one workpiece corner point corresponding to the vehicle clamp in the to-be-recognized image and an actual descriptor of each workpiece corner point; calculating the similarity between each actual descriptor and the theoretical descriptor; calculating the offset distance between the actual position information and the theoretical position information of the workpiece angular point corresponding to the maximum similarity in the to-be-recognized image; and generating an offset detection result of the vehicle clamp based on the offset distance.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a vehicle fixture offset detection method and a server. Background Art

[0002] Fixture positioning measurement is a key link in the field of industrial manufacturing, which is related to product quality and production efficiency. In the industrial production process, if the position of the fixture is offset, it will cause damage to the workpiece at the least, or even cause the production line to stop for maintenance, seriously affecting production efficiency. Therefore, fixture positioning is an important means to ensure production quality and improve production efficiency.

[0003] Therefore, how to improve the positioning accuracy of vehicle fixtures has become an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a vehicle fixture offset detection method and a server, which are used to solve the problem of how to improve the positioning accuracy of the vehicle fixture.

[0005] In a first aspect, the present application provides a vehicle fixture offset detection method, comprising: obtaining an image to be identified that includes a vehicle on a vehicle fixture fixed on a flexible drive conveying system in a preset area, and configuration information of standard corner points corresponding to the preset area; wherein the configuration information includes theoretical position information of the vehicle fixture in a theoretical identification image, and theoretical descriptors of standard corner points of the vehicle fixture in the theoretical identification image; performing image recognition on the image to be identified, determining at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified, and an actual descriptor of each workpiece corner point; calculating the similarity between each actual descriptor and the theoretical descriptor; calculating the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified; and generating an offset detection result of the vehicle fixture based on the offset distance.

[0006] In some feasible examples, image recognition is performed on an image to be identified to determine at least one workpiece corner point corresponding to a vehicle fixture in the image to be identified, as well as an actual descriptor of each workpiece corner point, including: performing image recognition on the image to be identified to determine a fixture image corresponding to the vehicle fixture in the image to be identified; preprocessing the fixture image to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image denoising; performing corner point detection on the preprocessed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified; performing the following operations on each workpiece corner point: taking the workpiece corner point as the center and a preset distance as the radius, determining the gradient features of all pixel points within the radius; determining the main direction of the workpiece corner point based on the gradient features; constructing a corner point feature matrix of the workpiece corner point based on the main direction and the gradient features; and using the corner point feature matrix as the actual descriptor of the workpiece corner point.

[0007] In some feasible examples, image recognition is performed on an image to be recognized to determine a fixture image corresponding to a vehicle fixture in the image to be recognized, including: inputting the image to be recognized into an image recognition model for image recognition to determine a fixture image corresponding to the vehicle fixture in the image to be recognized; wherein the image recognition model is trained by historical images to be recognized and fixture images corresponding to vehicle fixtures in historical images to be recognized.

[0008] In some feasible examples, calculating the similarity between each actual descriptor and the theoretical descriptor includes: performing the following operations for each actual descriptor: calculating the Euclidean distance between the actual descriptor and the theoretical descriptor, and using the Euclidean distance as the similarity between the actual descriptor and the theoretical descriptor.

[0009] In some feasible examples, the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified is calculated, including: calculating the pixel distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified; converting the pixel distance from the image coordinate system to the world coordinate system to obtain the world coordinate system distance, and using the world coordinate system distance as the offset distance.

[0010] In a second aspect, the present application provides a server, comprising: an acquisition module, used to acquire an image to be identified in a preset area containing a vehicle on a vehicle fixture fixed on a flexible drive conveying system, and configuration information of standard corner points corresponding to the preset area; wherein the configuration information includes theoretical position information of the vehicle fixture in a theoretical identification image, and theoretical descriptors of standard corner points of the vehicle fixture in the theoretical identification image; a processing module, used to perform image recognition on the image to be identified acquired by the acquisition module, determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified, and an actual descriptor of each workpiece corner point; the processing module is also used to calculate the similarity between each actual descriptor and the theoretical descriptor; the processing module is also used to calculate the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified; the processing module is also used to generate an offset detection result of the vehicle fixture based on the offset distance.

[0011] In some implementable examples, a processing module is specifically used to perform image recognition on the image to be identified acquired by the acquisition module, and determine the fixture image corresponding to the vehicle fixture in the image to be identified; the processing module is specifically used to preprocess the fixture image to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image noise reduction; the processing module is specifically used to perform corner point detection on the preprocessed image, and determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified; the processing module is specifically used to perform the following operations on each workpiece corner point: taking the workpiece corner point as the center and a preset distance as the radius, determine the gradient features of all pixel points within the radius; based on the gradient features, determine the main direction of the workpiece corner point; based on the main direction and the gradient features, construct a corner point feature matrix of the workpiece corner point; and use the corner point feature matrix as the actual descriptor of the workpiece corner point.

[0012] In some implementable examples, the processing module is specifically used to input the image to be identified acquired by the acquisition module into the image recognition model for image recognition, and determine the fixture image corresponding to the vehicle fixture in the image to be identified; wherein the image recognition model is trained by historical images to be identified and fixture images corresponding to the vehicle fixtures in the historical images to be identified.

[0013] In some implementable examples, the processing module is specifically configured to perform the following operations for each actual descriptor: calculating the Euclidean distance between the actual descriptor and the theoretical descriptor, and using the Euclidean distance as the similarity between the actual descriptor and the theoretical descriptor.

[0014] In a third aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above method.

[0015] Compared with the prior art, the technical solution provided by the present invention has the following advantages:

[0016] The vehicle fixture offset detection method provided by the present disclosure can obtain the configuration information of the standard corner points corresponding to the preset area by pre-shooting the theoretical recognition image including the vehicle fixture in the preset area. Afterwards, by obtaining the image to be identified in the preset area including the vehicle on the vehicle fixture fixed on the flexible drive conveying system, and the configuration information of the standard corner points corresponding to the preset area; thereby, image recognition can be performed on the image to be identified, and at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified, as well as the actual descriptor of each workpiece corner point can be determined; the similarity between each actual descriptor and the theoretical descriptor is calculated; the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified is calculated; based on the offset distance, the offset detection result of the vehicle fixture is generated. In this way, it is possible to determine whether the vehicle fixture is offset based on the offset detection result, thereby avoiding the problem of reduced positioning accuracy of the vehicle fixture due to the offset of the vehicle fixture, and solving the problem of how to improve the positioning accuracy of the vehicle fixture. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 exemplarily shows one of the flow charts of a vehicle fixture offset detection method provided in the first embodiment;

[0020] Figure 2 exemplarily shows one of the scenario schematic diagrams of a vehicle fixture offset detection method provided in the first embodiment;

[0021] Figure 3 FIG. 2 exemplarily shows a second scenario diagram of a vehicle fixture offset detection method provided in the first embodiment;

[0022] Figure 4 FIG. 2 exemplarily shows a second flow chart of a vehicle fixture offset detection method provided in the first embodiment;

[0023] Figure 5 FIG. 3 exemplarily shows a flow chart of a vehicle fixture offset detection method provided in the first embodiment;

[0024] Figure 6FIG4 is a flow chart of a vehicle fixture offset detection method provided in the first embodiment;

[0025] Figure 7 FIG. 5 exemplarily shows a flow chart of a vehicle fixture offset detection method provided in the first embodiment;

[0026] Figure 8 One of the structural diagrams of the server provided in the second embodiment is exemplarily shown in FIG.

[0027] Fig. 9 The second structural diagram of the server provided in the second embodiment is exemplarily shown in FIG. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0031] The technical solutions of the present application and the technical solutions of the present application are described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In the description of the present application, unless otherwise clearly specified and limited, each term should be understood in a broad sense in the art. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0032] YOLOv8 in the disclosed embodiment is the eighth version of the YOLO (You Only Look Once) series and is a target detection algorithm.

[0033] The Harris algorithm in the disclosed embodiment is a corner detection algorithm, which is developed by Chris Harris and Mike Stephens based on the Moravec algorithm.

[0034] The FAST algorithm in the embodiment of the present disclosure is an algorithm for corner point detection, and its full name is Features from Accelerated Segment Test. The algorithm detects corner points by comparing the difference in pixel values ​​between the detection point in the image and its surrounding area.

[0035] The 2-norm space in the embodiment of the present disclosure refers to a space in which the square root of the sum of the squares of the vector elements is used as the length or size of the vector in the Euclidean space. The 2-norm space is also called the Euclidean norm or L2 norm space.

[0036] The image neighborhood in the embodiment of the present disclosure refers to a set of pixels in an image, centered at a certain pixel and including a certain range of pixels around it.

[0037] Embodiment 1

[0038] Figure 1 A schematic diagram of a vehicle fixture offset detection method flow chart is shown in FIG. 1 . The execution subject of this example may be a server, such as Figure 1 As shown, the method includes:

[0039] S11. Obtain an image to be identified in a preset area that includes a vehicle on a vehicle fixture fixed on a flexible drive conveying system, and configuration information of standard corner points corresponding to the preset area; wherein the configuration information includes theoretical position information of the vehicle fixture in a theoretical identification image, and theoretical descriptors of standard corner points of the vehicle fixture in the theoretical identification image.

[0040] In some examples, the Flexible Drive System (FDS) is used in scenarios such as Figure 2 As shown, the assembly parts produced in the automobile production line are placed in FDS ( Figure 2 The object in the dotted box is transported to the next station for further processing. During the placement process, FDS Figure 3The positioning pins shown are used to position the assembly parts to fix their relative posture with themselves, so as to ensure that they enter the next process in an ideal posture. However, since the FDS itself may be deformed and the connectors may be loose, the assembly parts cannot enter the positioning pins, so the grabbing robot at the lower part port cannot smoothly remove the assembly parts from the FDS, which will seriously affect the production efficiency of the line. To this end, the vehicle fixture offset detection method provided by the embodiment of the present disclosure, when the FDS moves to the preset area, the image acquisition device, such as a camera, set in the preset area will shoot the preset area with a target posture, thereby obtaining an image to be identified of the vehicle fixed on the vehicle fixture of the flexible drive conveying system. At the same time, in order to facilitate the positioning of the position of the vehicle fixture, an initial image is shot by the camera with a target posture as a theoretical recognition image, and stored in the memory of the server. Since the theoretical recognition image contains the position information of each vehicle fixture in the theoretical recognition image when the vehicle fixture of the flexible drive conveying system is fixed with a vehicle, and the theoretical descriptor of the standard corner point corresponding to each vehicle fixture. In this way, in the subsequent use process, the offset detection result of the vehicle fixture can be determined based on the theoretical recognition image and the image to be recognized. Furthermore, based on the offset detection result, it can be determined whether the vehicle fixture is offset, thereby ensuring the positioning accuracy of the vehicle fixture.

[0041] In some examples, the theoretical descriptor is used to describe information about points surrounding a standard corner point of a vehicle angle, such as an image neighborhood of a standard corner point of a vehicle fixture in a theoretical recognition image.

[0042] In some examples, the image acquisition device is disposed above the FDS, and when the FDS reaches the position where the camera is located, an image of the FDS positioning pin taken by the camera is acquired.

[0043] S12, performing image recognition on the image to be recognized, and determining at least one workpiece corner point corresponding to the vehicle fixture in the image to be recognized, and an actual descriptor of each workpiece corner point.

[0044] In some examples, the image to be identified can be input into the recognition model for image recognition to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified, as well as the actual descriptor of each workpiece corner point. The training process of the recognition model is as follows:

[0045] Acquire first training sample data and a first labeling result of the first training sample data; the first training sample data includes a historical recognition image, and the first labeling result includes at least one workpiece corner point corresponding to the historical recognition image, and a theoretical descriptor of each workpiece corner point;

[0046] Inputting the first training sample data into the first neural network model for learning, and obtaining a first prediction result of the first neural network model on the first training sample data;

[0047] Based on the first prediction result and the first labeling result, the network parameters of the first neural network model are adjusted until the first neural network model converges to obtain a recognition model.

[0048] Alternatively, the image to be identified is subjected to image recognition to determine the fixture image corresponding to the vehicle fixture in the image to be identified; the fixture image is preprocessed to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image noise reduction. Corner point detection is performed on the preprocessed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified.

[0049] S13. Calculate the similarity between each actual descriptor and the theoretical descriptor.

[0050] In some examples, a preset distance between the actual descriptor and the theoretical descriptor may be used as the similarity between the actual descriptor and the theoretical descriptor; wherein the preset distance includes Euclidean distance, Manhattan distance, and Chebyshev distance.

[0051] In some examples, the cosine similarity between the actual descriptor and the theoretical descriptor may be used as the similarity between the actual descriptor and the theoretical descriptor.

[0052] S14, calculating the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified.

[0053] In some examples, an image coordinate system is established with the lower left vertex of the image to be identified as the origin, the direction perpendicular to the upper left vertex of the image to be identified as the x-axis direction, and the direction perpendicular to the lower right vertex of the image to be identified as the y-axis direction. In this way, the pixel points belonging to the working corner point can be determined. The position information corresponding to all the pixel points of the working corner point is averaged to obtain the actual position information of the working corner point in the image to be identified.

[0054] Exemplarily, assuming that there are three pixels belonging to the working corner point, namely pixel 1, pixel 2 and pixel 3, the position information of pixel 1 in the image coordinate system is (a, b), the position information of pixel 2 in the image coordinate system is (a1, b1), and the position information of pixel 3 in the image coordinate system is (a2, b2), the average value of the position information corresponding to all pixels of the working corner point includes: Therefore, the actual position information of the working corner point in the image to be identified is (c, d).

[0055] Similarly, the image coordinate system is established with the lower left vertex of the theoretical recognition image as the origin, the direction perpendicular to the upper left vertex of the theoretical recognition image as the x-axis direction, and the direction perpendicular to the lower right vertex of the theoretical recognition image as the y-axis direction. In this way, the pixel points belonging to the working corner point can be determined. The position information corresponding to all the pixels of the working corner point is averaged to obtain the theoretical position information of the working corner point in the theoretical recognition image.

[0056] S15. Generate an offset detection result of the vehicle fixture based on the offset distance.

[0057] In some examples, when the offset distance is equal to 0, the offset detection result is determined to be not offset; when the offset distance is not equal to 0, the offset detection result is determined to be offset. When the offset detection result is offset, the server can generate a prompt message indicating that the vehicle fixture is offset, so as to prompt the user to correct the vehicle fixture in time to ensure the normal operation of the vehicle fixture.

[0058] As can be seen from the above, the vehicle fixture offset detection method provided by the present disclosure can obtain the configuration information of the standard corner points corresponding to the preset area by pre-shooting the theoretical recognition image including the vehicle fixture in the preset area. Afterwards, by obtaining the image to be identified in the preset area including the vehicle on the vehicle fixture fixed on the flexible drive conveying system, and the configuration information of the standard corner points corresponding to the preset area; thereby, image recognition can be performed on the image to be identified, and at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified, as well as the actual descriptor of each workpiece corner point can be determined; the similarity between each actual descriptor and the theoretical descriptor is calculated; the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified is calculated; based on the offset distance, the offset detection result of the vehicle fixture is generated. In this way, it is possible to determine whether the vehicle fixture is offset based on the offset detection result, thereby avoiding the problem of reduced positioning accuracy of the vehicle fixture due to the offset of the vehicle fixture, and solving the problem of how to improve the positioning accuracy of the vehicle fixture.

[0059] In some possible implementation examples, combined with Figure 1 ,like Figure 4 As shown, the above S12 can be specifically implemented through the following S120-S123.

[0060] S120 , performing image recognition on the image to be recognized, and determining a fixture image corresponding to the vehicle fixture in the image to be recognized.

[0061] In some examples, performing image recognition on the image to be recognized and determining a fixture image corresponding to the vehicle fixture in the image to be recognized includes:

[0062] Perform image recognition on the image to be recognized to determine the foreground image and the background image in the image to be recognized. Perform image segmentation on the foreground image to obtain at least one segmented image. Calculate the similarity between the segmented image and the preset image of the vehicle fixture. Use the segmented image corresponding to the maximum similarity as the fixture image of the vehicle fixture.

[0063] Alternatively, the image to be identified is input into an image recognition model for image recognition, and the fixture image corresponding to the vehicle fixture in the image to be identified is determined; wherein the image recognition model is trained by historical images to be identified and fixture images corresponding to the vehicle fixture in the historical images to be identified. The training process of the image recognition model includes:

[0064] Acquire second training sample data and a second marking result of the second training sample data; the second training sample data includes a historical recognition image, and the second marking result includes a fixture image corresponding to a vehicle fixture corresponding to the historical recognition image.

[0065] Inputting the second training sample data into the second neural network model for learning, and obtaining a second prediction result of the second neural network model for the second training sample data;

[0066] Based on the second prediction result and the second labeling result, the network parameters of the second neural network model are adjusted until the second neural network model converges to obtain an image recognition model.

[0067] In some examples, the second neural network model can be YOLOv8.

[0068] In some examples, since the image to be identified contains tens of millions of pixels, directly locating the working corner points accurately will cause great computational complexity. Therefore, image recognition is first performed on the initial image to obtain the approximate area image where the vehicle fixture is located, and then the subsequent working corner points are accurately located.

[0069] S121, preprocessing the fixture image to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image noise reduction.

[0070] In some examples, a Laplace filter may be used to perform edge enhancement processing on the extracted fixture image to obtain an edge enhanced image; then, a bilateral filter may be used to eliminate image noise in the edge enhanced image to obtain a denoised image, and the denoised image may be used as a preprocessed image.

[0071] S122, performing corner point detection on the preprocessed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified.

[0072] In some examples, a Shi-Tomasi corner detector may be used to perform corner detection on the preprocessed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified.

[0073] In some examples, workpiece corner extraction may also be performed using a corner extraction algorithm such as Harris algorithm and FAST algorithm.

[0074] In some examples, it is assumed that there are N vehicle fixtures in the FDS in the image to be identified. For example, the vehicle fixture is a locating pin. The following processing is for all N locating pins in parallel. For the sake of simplicity, the algorithm described in this article takes the nth locating pin as an example, and the fixture image of the nth locating pin is named P n . Where n∈[1,N], and n is an integer.

[0075] S3.1: Construct the Laplacian operator:

[0076]

[0077] S3.2: Use the Laplacian operator to calculate the fixture image P of the nth locating pin n To process:

[0078]

[0079] Among them, c is a constant and * is the image convolution operation.

[0080] S3.3: Use bilateral filter to Processing is performed to reduce noise while protecting edge information. For each pixel in , the following operations are performed:

[0081]

[0082] in,

[0083]

[0084] Among them, x represents The horizontal coordinate of the pixel point in the image coordinate system, y represents The vertical coordinate of the pixel point in the image coordinate system, and The variances are preset.

[0085] S3.4: Use the Sobel operator (Sobel) to calculate The gradient I of each pixel in the horizontal direction (i.e., the x-axis direction in the image coordinate system) xThe gradient I in the vertical direction (i.e., the y-axis direction in the image coordinate system) y (x, y), the principle is as follows:

[0086]

[0087] Among them, G x represents the Sobel operator in the x-axis direction in the image coordinate system, G y Represents the Sobel operator in the y-axis direction in the image coordinate system.

[0088] S3.5: Each pixel in constructs the M matrix in the Shi-Tomasi corner detector:

[0089]

[0090] Among them, ∑ is the neighborhood of the pixel point, and the neighborhood of the pixel point refers to the area corresponding to the circle with the pixel point as the center and the target distance as the radius.

[0091] S3.4: Extract The eigenvalues ​​λ1 and λ2 of the M matrix of each pixel in are calculated. If the minimum value among the eigenvalues ​​is greater than a given threshold η, this pixel is set as a working corner point.

[0092] S3.5: Determine the neighborhood of the working corner point. The neighborhood of the working corner point refers to the area corresponding to the circle drawn with the working corner point as the center and the preset distance as the radius; delete the corner point whose minimum eigenvalue in the neighborhood of the working corner point is smaller than the minimum eigenvalue of the working corner point corresponding to the neighborhood, and obtain the corner point set C n ={c (n，1) , …, c (n，K)}.

[0093] in, is the position information of the kth working corner point in the image coordinate system, and K is the total number of working corner points.

[0094] It should be noted that the position information includes coordinates in the image coordinate system.

[0095] In this way, the corner point set C can be n Each working corner point in is used as at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified.

[0096] S123, performing the following operations on each workpiece corner point: taking the workpiece corner point as the center and a preset distance as the radius, determining the gradient features of all pixel points within the radius; constructing a corner point feature matrix of the workpiece corner point based on the gradient features; and using the corner point feature matrix as the actual descriptor of the workpiece corner point.

[0097] In some examples, combined with the example given in S122 above, this section will calculate the corner point set C n The actual descriptor of each working corner point in is used as the corner point set C for the convenience of subsequent description. n Working corner point c in (n , k) Take the calculation as an example, the specific implementation process is as follows:

[0098] S4.1: Take the working corner c (n , k) As the center, the working corner point c (n , k) The corresponding neighborhood is divided into m×m sub-regions; each sub-region is a square with a side length of l. In addition, assuming that the working corner point c (n，k) The set of all pixels in the corresponding neighborhood is ∑ c , calculated from S3.4 x with I y The working corner c can be obtained from (n , k) The gradient value of each pixel in the corresponding neighborhood.

[0099] S4.2: Calculate the working corner c (n , k) The direction and amplitude of each pixel in the corresponding neighborhood, taking the pixel point P(x, y) as an example:

[0100]

[0101] Among them, arg(P(x, y)) represents the direction of the pixel point P(x, y), amp(P(x, y)0 represents the amplitude of the pixel point P(x, y), and the unit of arg(P(x, y)) is angle system.

[0102] S4.3: Constructing a multidimensional vector C (n , k) ∈R m×m×8 is the working corner point c (n , k) The characteristic vector of each working corner point c (n，k) The values ​​of each sub-region in the corresponding neighborhood in 8 directions (every 45 degrees is an interval) are calculated as follows:

[0103] Among them, the working corner c (n，k) The vector is initially a zero vector, and each entry is initially zero.

[0104] For P(x,y)in∑ c ;

[0105] Among them, assume that (x0, y0) is the working corner point c (n，k) The coordinates of the upper left corner pixel of the corresponding neighborhood are calculated as follows:

[0106]

[0107] in, is the working corner point c (n，k) Position information in the image coordinate system, Indicates that m is rounded down.

[0108]

[0109] m 11 =m1×y diff , m 10 =m1×(1-y diff );

[0110] m 01 =m0×y diff , m 00 =m0×(1-y diff );

[0111] m 111 =m 11 ×θ diff , m 110 =m 11 ·(1-θ diff );

[0112] m 101 =m 10 ×θ diff , m 100 =m 10 ·(1-θ diff );

[0113] m 011 =m 01 ×θ diff , m 010 =m 01 ·(1-θ diff );

[0114] m 001 =m 00 ×θ diff , m 000 =m 00 ·(1-θ diff );

[0115] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0116]

[0117] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0118]

[0119] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0120]

[0121] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0122]

[0123] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0124]

[0125] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0126]

[0127] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0128]

[0129] Sub-area exist The corresponding values ​​in the direction are updated as follows:

[0130]

[0131] Wherein, A+=B represents the sum of A and B, a∈[1,8], and a is an integer.

[0132] After that, the working corner point c (n , k) The one-dimensional vector composed of the values ​​of each sub-region in each direction in the corresponding neighborhood is used as the working corner point c (n , k) The actual descriptor.

[0133] Or, with the workpiece corner c (n ,k) As the center, with the preset distance as the radius, determine the pixel values ​​of all pixels within the radius. Based on the pixel values ​​of all pixels within the radius, form a target matrix and use the target matrix as the working corner point c (n , k) The actual descriptor.

[0134] It should be noted that due to c n The actual calculation process of the descriptor of other working corner points in c n Working corner point c in (n , k) The calculation process of the actual descriptor is similar and will not be repeated here.

[0135] As can be seen from the above, the vehicle fixture offset detection method provided by the present disclosure can obtain the configuration information of the standard corner points corresponding to the preset area by pre-shooting the theoretical recognition image including the vehicle fixture in the preset area. Afterwards, the image to be recognized is subjected to image recognition to determine the fixture image corresponding to the vehicle fixture in the image to be recognized; the fixture image is pre-processed to obtain a pre-processed image; wherein the pre-processing includes any one of edge enhancement and image noise reduction; corner point detection is performed on the pre-processed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be recognized; the following operations are performed on each workpiece corner point: with the workpiece corner point as the center and the preset distance as the radius, the gradient features of all pixel points within the radius are determined; based on the gradient features, the main direction of the workpiece corner point is determined; based on the main direction and the gradient features, the corner point feature matrix of the workpiece corner point is constructed; and the corner point feature matrix is ​​used as the actual descriptor of the workpiece corner point. In this way, similarity calculation can be performed based on the actual descriptor corresponding to the working corner point and the theoretical descriptor of the standard corner point, so that the best matching workpiece corner point can be found. Afterwards, by calculating the offset distance between the matched workpiece corner point and the standard corner point, the offset detection result of the vehicle fixture can be generated based on the offset distance. Afterwards, it can be determined whether the vehicle fixture is offset based on the offset detection result, thereby avoiding the problem of reduced positioning accuracy of the vehicle fixture caused by the offset of the vehicle fixture, and solving the problem of how to improve the positioning accuracy of the vehicle fixture.

[0136] In some possible implementation examples, combined with Figure 4 ,like Figure 5 As shown, the above S120 can be specifically implemented through the following S1200.

[0137] S1200, inputting the image to be recognized into an image recognition model for image recognition, and determining a fixture image corresponding to the vehicle fixture in the image to be recognized; wherein the image recognition model is trained by historical images to be recognized and fixture images corresponding to the vehicle fixtures in the historical images to be recognized.

[0138] As can be seen from the above, the vehicle fixture offset detection method provided by the present disclosure is pre-trained by using historical images to be identified and fixture images corresponding to the vehicle fixtures in the historical images to be identified, so that the image recognition model can identify the fixture images corresponding to the vehicle fixtures in the images to be identified. In this way, the amount of calculation during precise positioning can be reduced, the generation time of offset detection results can be improved, and the normal production can be ensured.

[0139] In some possible implementation examples, combined with Figure 1 ,like Figure 6 As shown, the above S13 can be specifically implemented through the following S130.

[0140] S130 , performing the following operations for each actual descriptor: calculating the Euclidean distance between the actual descriptor and the theoretical descriptor, and using the Euclidean distance as the similarity between the actual descriptor and the theoretical descriptor.

[0141] In some examples, the process of calculating the Euclidean distance between the actual descriptor and the theoretical descriptor is as follows:

[0142] S4.4: Assume that the standard corner point corresponding to the nth locating pin is c (n) , and its eigenvector is C (n) . Then, calculate the corner point set C n The eigenvectors C of all corner points in (n) With C (n) The distance in 2-norm space. Among them, with the eigenvector C (n) The corner point corresponding to the nearest feature vector is the target corner point of the current frame, which is represented by the corner point corner point The value-taking process can also be expressed as:

[0143]

[0144] In this way, the workpiece corner point corresponding to the maximum similarity is obtained.

[0145] As can be seen from the above, the vehicle fixture offset detection method provided by the present disclosure calculates the Euclidean distance between the actual descriptor and the theoretical descriptor, and uses the Euclidean distance as the similarity between the actual descriptor and the theoretical descriptor, so that the most matching workpiece corner point can be found based on the similarity. Afterwards, by calculating the offset distance between the matching workpiece corner point and the standard corner point, the offset detection result of the vehicle fixture can be generated based on the offset distance. Afterwards, it can be determined based on the offset detection result whether the vehicle fixture is offset, thereby avoiding the problem of reduced positioning accuracy of the vehicle fixture caused by the offset of the vehicle fixture, and solving the problem of how to improve the positioning accuracy of the vehicle fixture.

[0146] In some possible implementation examples, combined with Figure 1 ,like Figure 7 As shown, the above S14 can be specifically implemented through the following S140 and S141.

[0147] S140, calculating the pixel distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified.

[0148] In some examples, each preset area corresponds to a theoretical recognition image. In this way, when the vehicle fixed to the vehicle fixture of the flexible drive conveying system reaches the preset area, the image acquisition device will shoot an image to be recognized according to the target posture corresponding to the theoretical recognition image. In this way, by processing the image to be recognized, the workpiece corner point corresponding to the maximum similarity corresponding to the image to be recognized and the actual position information of the workpiece corner point corresponding to the maximum similarity in the image to be recognized are obtained. At the same time, the theoretical recognition image contains a standard corner point, and the standard corner point also has a theoretical position information in the theoretical recognition image. Since the actual position information and the theoretical position information are in the same image coordinate system, the pixel distance between the actual position information and the theoretical position information can be calculated, such as the horizontal distance of pixels in the x-axis direction, and the vertical distance of pixels in the y-axis direction.

[0149] S141 , converting the pixel distance from the image coordinate system to the world coordinate system to obtain the world coordinate system distance, and using the world coordinate system distance as the offset distance.

[0150] In some examples, since the pixel distance is described in pixels, it is not possible to intuitively express the distance that the vehicle fixture is offset. Therefore, the pixel distance needs to be converted from the image coordinate system to the world coordinate system, such as calculating the size of a single pixel point in the image coordinate system parallel to the plane where the positioning pin is located in the world coordinate system. In this way, the size of a single pixel in the image coordinate system in the world coordinate system is obtained. After that, the pixel distance is converted from the image coordinate system to the world coordinate system to obtain the world coordinate system distance, such as:

[0151]

[0152] Wherein, Δx represents the offset lateral distance in the world coordinate system distance, and Δy represents the offset longitudinal distance in the world coordinate system distance.

[0153] Afterwards, the offset lateral distance and the offset longitudinal distance are both taken as the offset distance. Thus, when both the offset lateral distance and the offset longitudinal distance are 0, it indicates that the vehicle fixture has not been offset, that is, the offset detection result is not offset. When one or more of the offset lateral distance and the offset longitudinal distance are not 0, it indicates that the vehicle fixture has been offset, that is, the offset detection result is offset.

[0154] As can be seen from the above, the vehicle fixture offset detection method provided by the present disclosure cannot intuitively represent the offset distance of the vehicle fixture because the image coordinate system is in pixels. Therefore, it is necessary to convert the pixel distance from the image coordinate system to the world coordinate system to obtain the world coordinate system distance. In this way, the offset distance of the vehicle fixture can be better represented, improving the user experience.

[0155] Embodiment 2

[0156] Figure 8 The structural diagram of the server provided in the second embodiment of the present application is exemplarily shown in FIG. Figure 8 As shown, the server includes: an acquisition module 81 and a processing module 82.

[0157] The acquisition module 81 is used to acquire an image to be identified of a vehicle in a preset area including a vehicle fixture fixed on a flexible drive conveying system, and configuration information of standard corner points corresponding to the preset area; wherein the configuration information includes theoretical position information of the vehicle fixture in a theoretical identification image, and theoretical descriptors of standard corner points of the vehicle fixture in the theoretical identification image;

[0158] The processing module 82 is used to perform image recognition on the image to be recognized obtained by the acquisition module 81, and determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be recognized, and an actual descriptor of each workpiece corner point;

[0159] The processing module 82 is also used to calculate the similarity between each actual descriptor and the theoretical descriptor;

[0160] The processing module 82 is further used to calculate the offset distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified;

[0161] The processing module 82 is further configured to generate an offset detection result of the vehicle fixture based on the offset distance.

[0162] In some implementable examples, the processing module 82 is specifically used to perform image recognition on the image to be identified acquired by the acquisition module 81, and determine the fixture image corresponding to the vehicle fixture in the image to be identified; the processing module 82 is specifically used to preprocess the fixture image to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image denoising; the processing module 82 is specifically used to perform corner point detection on the preprocessed image, and determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified; the processing module 82 is specifically used to perform the following operations on each workpiece corner point: taking the workpiece corner point as the center and a preset distance as the radius, determine the gradient features of all pixel points within the radius; based on the gradient features, determine the main direction of the workpiece corner point; based on the main direction and the gradient features, construct a corner point feature matrix of the workpiece corner point; and use the corner point feature matrix as the actual descriptor of the workpiece corner point.

[0163] In some implementable examples, the processing module 82 is specifically used to input the image to be identified acquired by the acquisition module 81 into the image recognition model for image recognition, and determine the fixture image corresponding to the vehicle fixture in the image to be identified; wherein the image recognition model is trained by historical images to be identified and fixture images corresponding to the vehicle fixtures in the historical images to be identified.

[0164] In some implementable examples, the processing module 82 is specifically configured to perform the following operations for each actual descriptor: calculating the Euclidean distance between the actual descriptor and the theoretical descriptor, and using the Euclidean distance as the similarity between the actual descriptor and the theoretical descriptor.

[0165] In some implementable examples, the processing module 82 is specifically used to calculate the pixel distance between the actual position information and the theoretical position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified; the processing module 82 is specifically used to convert the pixel distance from the image coordinate system to the world coordinate system, obtain the world coordinate system distance, and use the world coordinate system distance as the offset distance.

[0166] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.

[0167] Of course, the server provided by the embodiment of the present invention includes but is not limited to the above modules, for example, the server may also include a storage module 83. The storage module 83 may be used to store program codes of the server, and may also be used to store data generated during the operation of the server, such as diagnostic data.

[0168] Fig. 9 A schematic diagram of the structure of a server provided in an embodiment of the present invention, such as Fig. 9As shown, the server may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .

[0169] Combine the following Fig. 9 A detailed introduction to each component of the server:

[0170] The processor 51 is the control center of the server, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0171] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Fig. 9 Also, as an embodiment, the server may include multiple processors, such as Fig. 9 51 and 55 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0172] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0173] In a specific implementation, the memory 52 is used to store the data of the present invention and execute the software program of the present invention. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.

[0174] The communication interface 53 uses any transceiver-like device to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, the cloud, etc. The communication interface 53 may include an acquisition module to implement the acquisition function.

[0175] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0176] As an example, combining Figure 8The function implemented by the acquisition module 81 of the server is the same as Fig. 9 The function of the communication interface 53 in the server is the same as that of the processing module 82 in the server. Fig. 9 The functions of the processor 51 in the server are the same as those of the storage module 83 in the server. Fig. 9 The function of the memory 52 in is the same.

[0177] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method in any embodiment.

[0178] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle fixture offset detection method, characterized in that: include: Acquire an image to be identified of a vehicle in a preset area including a vehicle fixture fixed on a flexible drive conveying system, and configuration information of standard corner points corresponding to the preset area; wherein the configuration information includes theoretical position information of the vehicle fixture in a theoretical identification image, and theoretical descriptors of standard corner points of the vehicle fixture in the theoretical identification image; Performing image recognition on the image to be recognized, determining at least one workpiece corner point corresponding to the vehicle fixture in the image to be recognized, and an actual descriptor of each of the workpiece corner points; Calculating the similarity between each of the actual descriptors and the theoretical descriptors; Calculate the offset distance between the actual position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified and the theoretical position information; Based on the offset distance, an offset detection result of the vehicle fixture is generated.

2. The vehicle fixture offset detection method according to claim 1, characterized in that: The performing image recognition on the image to be recognized, and determining at least one workpiece corner point corresponding to the vehicle fixture in the image to be recognized, and an actual descriptor of each workpiece corner point, includes: Performing image recognition on the image to be recognized, and determining a fixture image corresponding to the vehicle fixture in the image to be recognized; Preprocessing the fixture image to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image noise reduction; Performing corner point detection on the preprocessed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified; The following operations are performed on each of the workpiece corner points: taking the workpiece corner point as the center and a preset distance as the radius, determining the gradient features of all pixel points within the radius; based on the gradient features, determining the main direction of the workpiece corner point; based on the main direction and the gradient features, constructing a corner point feature matrix of the workpiece corner point; and using the corner point feature matrix as an actual descriptor of the workpiece corner point.

3. The vehicle fixture offset detection method according to claim 2, characterized in that: The performing image recognition on the image to be recognized to determine the fixture image corresponding to the vehicle fixture in the image to be recognized includes: The image to be identified is input into an image recognition model for image recognition, and a fixture image corresponding to the vehicle fixture in the image to be identified is determined; wherein the image recognition model is trained by historical images to be identified and fixture images corresponding to the vehicle fixtures in the historical images to be identified.

4. The vehicle fixture offset detection method according to claim 1, characterized in that: The calculating the similarity between each of the actual descriptors and the theoretical descriptors comprises: For each actual descriptor, the following operations are performed: the Euclidean distance between the actual descriptor and the theoretical descriptor is calculated, and the Euclidean distance is used as the similarity between the actual descriptor and the theoretical descriptor.

5. The vehicle fixture offset detection method according to claim 1, characterized in that: The offset distance between the actual position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified and the theoretical position information includes: Calculate the pixel distance between the actual position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified and the theoretical position information; The pixel distance is converted from the image coordinate system to the world coordinate system to obtain the world coordinate system distance, and the world coordinate system distance is used as the offset distance.

6. A server, characterized in that: include: An acquisition module is used to acquire an image to be identified of a vehicle on a vehicle fixture fixed on a flexible drive conveying system in a preset area, and configuration information of standard corner points corresponding to the preset area; wherein the configuration information includes theoretical position information of the vehicle fixture in a theoretical identification image, and theoretical descriptors of standard corner points of the vehicle fixture in the theoretical identification image; A processing module, configured to perform image recognition on the image to be recognized acquired by the acquisition module, and determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be recognized, and an actual descriptor of each of the workpiece corner points; The processing module is further used to calculate the similarity between each of the actual descriptors and the theoretical descriptors; The processing module is further used to calculate the offset distance between the actual position information of the workpiece corner point corresponding to the maximum similarity in the image to be identified and the theoretical position information; The processing module is further configured to generate an offset detection result of the vehicle fixture based on the offset distance.

7. The server according to claim 6, characterized in that: The processing module is specifically used to perform image recognition on the image to be recognized acquired by the acquisition module, and determine a fixture image corresponding to the vehicle fixture in the image to be recognized; The processing module is specifically used to preprocess the fixture image to obtain a preprocessed image; wherein the preprocessing includes any one of edge enhancement and image noise reduction; The processing module is specifically used to perform corner point detection on the pre-processed image to determine at least one workpiece corner point corresponding to the vehicle fixture in the image to be identified; The processing module is specifically used to perform the following operations on each of the workpiece corner points: taking the workpiece corner point as the center and a preset distance as the radius, determining the gradient features of all pixel points within the radius; determining the main direction of the workpiece corner point based on the gradient features; constructing a corner point feature matrix of the workpiece corner point based on the main direction and the gradient features; and using the corner point feature matrix as an actual descriptor of the workpiece corner point.

8. The server according to claim 6, characterized in that: The processing module is specifically used to input the image to be identified acquired by the acquisition module into the image recognition model for image recognition, and determine the fixture image corresponding to the vehicle fixture in the image to be identified; wherein the image recognition model is trained by historical images to be identified and fixture images corresponding to the vehicle fixtures in the historical images to be identified.

9. The server according to claim 6, characterized in that: The processing module is specifically used to perform the following operations for each actual descriptor: calculating the Euclidean distance between the actual descriptor and the theoretical descriptor, and using the Euclidean distance as the similarity between the actual descriptor and the theoretical descriptor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

Images