Image measurement method, system and platform based on multi-feature fusion and AI automatic point collection

Abstract

The invention discloses an image measurement method, system and platform based on multi-feature fusion and AI automatic point collection. The method comprises the following steps: generating and acquiring first data corresponding to a to-be-measured product, and preprocessing the first data; wherein the first data is product image data; respectively generating corresponding second data and third data based on the first data in combination with a CSP backbone network structure and a self-adaptive anchor frame; generating corresponding fourth data based on a multi-task learning framework, and marking and processing the second data according to the fourth data to generate corresponding fifth data; according to the fifth data, constructing a corresponding measurement element model, and based on the measurement element model, generating corresponding sixth data; wherein the sixth data is measurement data corresponding to the product to be measured, and the system and the platform corresponding to the method realize high-efficiency and high-precision automatic measurement by combining an AI automatic point collection technology with a multi-feature fusion algorithm.

Description

Technical Field

[0001] The present invention belongs to the technical field of image measurement and processing, and specifically relates to an image measurement method, system and platform based on multi-feature fusion and AI automatic point collection. Background Art

[0002] As an important means of non-contact precision measurement, image measurement technology has been widely used in the field of industrial manufacturing. Traditional image measuring instruments usually rely on manual mouse point sampling to complete the measurement of complex surface features, such as irregular contours, tiny holes or complex curves.

[0003] For manual point collection, users need to click on the measurement points one by one, which is cumbersome and time-consuming. Especially in large-scale measurement tasks, the efficiency is difficult to meet production needs, that is, there is a problem of low efficiency and technical defects; in addition, manual point collection is greatly affected by the operator's experience and status, and it is easy to introduce human errors, resulting in inconsistent measurement results, that is, there is unstable accuracy; there is also poor adaptability. For complex surface features (such as curved surfaces, tiny structures, etc.), manual point collection is difficult to accurately capture feature points, resulting in reduced measurement accuracy.

[0004] In the existing technology, although some image measuring instruments have introduced automatic measurement functions, their scope of application is limited. They are usually only applicable to simple geometric features (such as straight lines, circles, etc.) and cannot effectively process complex surface features. In addition, the existing automatic measurement methods lack support for multi-feature fusion, making it difficult to fully utilize the multi-scale information in the image, resulting in insufficient measurement accuracy and robustness.

[0005] Therefore, in view of the above technical problems of low efficiency, difficulty in ensuring accuracy and poor adaptability of manual point collection, it is urgent to design and develop an image measurement method, system and platform based on multi-feature fusion and AI automatic point collection. Summary of the invention

[0006] In order to overcome the shortcomings and difficulties of the above-mentioned prior art, the present invention provides an image measurement method, system and platform based on multi-feature fusion and AI automatic point sampling, which solves the problems of low efficiency, unstable accuracy and poor adaptability of traditional manual point sampling measurement methods.

[0007] The first object of the present invention is to provide an image measurement method based on multi-feature fusion and AI automatic point collection; the second object of the present invention is to provide an image measurement system based on multi-feature fusion and AI automatic point collection; the third object of the present invention is to provide an image measurement platform based on multi-feature fusion and AI automatic point collection; The first object of the present invention is achieved in that the method comprises the following steps: Generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; Based on the first data, in combination with the CSP backbone network structure and the adaptive anchor frame, corresponding second data and third data are generated respectively; wherein the second data is image feature data; and the third data is anchor frame size data; Generate corresponding fourth data based on a multi-task learning framework, and mark and process the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; According to the fifth data, a corresponding measurement element model is constructed, and based on the measurement element model, corresponding sixth data is generated; wherein the sixth data is measurement data corresponding to the product to be measured.

[0008] Further, the generating and acquiring first data corresponding to the workpiece to be measured and preprocessing the first data also includes: Generate and obtain seventh data corresponding to the product to be measured and processed; wherein the seventh data is activation control data for starting to collect product image data; According to the seventh data, combined with data enhancement technology, the first data is preprocessed in real time to generate corresponding eighth data; wherein the preprocessing includes random scaling, cropping, flipping and translation; the eighth data is diversified training data corresponding to the image data.

[0009] Further, the generating of corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame also includes: Combined with the CSP backbone network structure, the first data is sequentially split and merged; A data set corresponding to the first data is created, and the data set is analyzed and processed to generate third data corresponding to the data set; wherein the data set includes image data of the product to be tested and corresponding annotation information data.

[0010] Further, the generating of corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame also includes: Based on the CSP backbone network structure, second data corresponding to the first data is calculated and generated; wherein the calculation formula is as follows: (1) In the formula, is the fused feature; It is the feature map of different stages; Combined with the adaptive learning algorithm, the anchor box size and loss function weight of the model are dynamically adjusted; the calculation formula of the anchor box size is as follows: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; It is the intersection and comparison.

[0011] Furthermore, the method of generating corresponding fourth data based on a multi-task learning framework, and marking and processing the second data according to the fourth data to generate corresponding fifth data further includes: Combined with the multi-task learning framework, the corresponding classification loss data, regression loss data and confidence loss data are calculated and generated respectively; Based on the second data and in combination with the first data, the corresponding fifth data is calculated and generated; wherein the calculation formula is as follows: (4) In the formula, For automatic sampling point location; For location confidence level; For location With real frame The intersection and union ratio of The area in the image where points are to be collected.

[0012] Further, the constructing a corresponding measurement element model according to the fifth data, and generating corresponding sixth data based on the measurement element model, further includes: According to the fifth data, construct at least one corresponding measurement element; wherein the measurement element includes a line, an arc and a distance; Combined with stochastic gradient descent or Adam optimizer, the convolutional neural network-based object detection model or key point detection model is trained and optimized respectively.

[0013] The second object of the present invention is achieved as follows: the system is used to implement the image measurement method based on multi-feature fusion and AI automatic point collection, and the system includes: A first data generating unit, used to generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; A second data generating unit is used to generate corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame, wherein the second data is image feature data; and the third data is anchor frame size data; A first data processing unit is used to generate corresponding fourth data based on a multi-task learning framework, and mark and process the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; The third data generating unit is used to construct a corresponding measurement element model according to the fifth data, and generate corresponding sixth data based on the measurement element model; wherein the sixth data is measurement data corresponding to the product to be measured.

[0014] Furthermore, the first data generating unit further includes: A first generating module is used to generate and obtain seventh data corresponding to the product to be measured and processed; wherein the seventh data is activation control data for starting to collect product image data; a first processing module, configured to pre-process the first data in real time according to the seventh data in combination with a data enhancement technology, and generate corresponding eighth data; wherein the pre-processing includes random scaling processing, cropping processing, flipping processing, and translation processing; the eighth data is diversified training data corresponding to the image data; and / or, the second data generating unit further includes: A second processing module is used to sequentially split and merge the first data in combination with the CSP backbone network structure; A third processing module, used to create a data set corresponding to the first data, and analyze and process the data set to generate third data corresponding to the data set; wherein the data set includes image data of the product to be tested and corresponding annotation information data; And / or, the first data processing unit further includes: The second generation module is used to combine the multi-task learning framework to calculate and generate corresponding classification loss data, regression loss data and confidence loss data respectively; The first calculation module is used to calculate and generate corresponding fifth data based on the second data and in combination with the first data; wherein the calculation formula is as follows: (4) In the formula, For automatic sampling point location; For location confidence level; For location With real frame The intersection and union ratio of is the area range of the image to be sampled; and / or, the third data generating unit further includes: A first construction module, configured to construct at least one corresponding measurement element according to the fifth data; wherein the measurement element includes a line, an arc and a distance; The fourth processing module is used to combine the stochastic gradient descent or Adam optimizer to respectively train and optimize the target detection model or the key point detection model based on the convolutional neural network.

[0015] Furthermore, the second data generating unit further includes: The second calculation module is used to calculate and generate second data corresponding to the first data based on the CSP backbone network structure; wherein the calculation formula is as follows: (1) In the formula, is the fused feature; are feature maps at different stages; the fifth processing module is used to dynamically adjust the anchor box size and loss function weight of the model in combination with the adaptive learning algorithm; wherein the calculation formula of the anchor box size is as follows: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; It is the intersection and comparison.

[0016] The third object of the present invention is achieved as follows: it includes a processor, a memory, and an image measurement platform control program based on multi-feature fusion and AI automatic point collection; wherein the image measurement platform control program based on multi-feature fusion and AI automatic point collection is executed in the processor, the image measurement platform control program based on multi-feature fusion and AI automatic point collection is stored in the memory, and the image measurement platform control program based on multi-feature fusion and AI automatic point collection implements the image measurement method based on multi-feature fusion and AI automatic point collection.

[0017] The present invention generates and obtains first data corresponding to a product to be measured and processed through a method, and pre-processes the first data; wherein the first data is product image data; based on the first data, and in combination with a CSP backbone network structure and an adaptive anchor frame, corresponding second data and third data are generated respectively; wherein the second data is image feature data; the third data is anchor frame size data; corresponding fourth data is generated based on a multi-task learning framework, and according to the fourth data, the second data is marked and processed to generate corresponding fifth data; wherein the fourth data is loss function data; the fifth data is sampling point location information data; according to the fifth data, a corresponding measurement element model is constructed, and based on the measurement element model, corresponding sixth data is generated; wherein the sixth data is measurement data corresponding to the product to be measured, as well as a system and platform corresponding to the method, through AI automatic sampling technology combined with a multi-feature fusion algorithm, efficient and high-precision automatic measurement is achieved.

[0018] That is to say, the solution of the present invention replaces manual operation with automatic point sampling by AI, which significantly reduces the measurement time and is particularly suitable for large-scale measurement tasks; it utilizes multi-feature fusion technology to fully extract multi-scale features in the image, and combines adaptive anchor frame calculation and multi-task learning framework to ensure high-precision positioning of the measurement points; through data enhancement technology and CSP backbone network, the model's recognition ability for complex surface features is improved, which is suitable for the measurement of complex structures such as irregular contours and tiny holes; the automatic point sampling process is completely completed by the AI ​​model, avoiding errors introduced by human operation and ensuring the consistency of the measurement results; through lightweight design and optimizer selection, the model parameters and calculation amount are reduced, the running speed is improved, and it is suitable for real-time measurement scenarios.

[0019] In other words, the solution of the present invention improves measurement accuracy, enhances measurement adaptability, reduces human errors, and makes the model lightweight; that is, it solves the problems of low efficiency, unstable accuracy, and poor adaptability of traditional manual point measurement methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A schematic diagram of the process steps of an image measurement method based on multi-feature fusion and AI automatic point collection according to the present invention; Figure 2 A schematic diagram of the conversion process steps of a manual mouse point measurement method based on multi-feature fusion and AI automatic point collection in the present invention from AI automatic point collection; Figure 3 A schematic diagram of the steps of a manual mouse point collection measurement process without using AI automatic point collection in an image measurement method based on multi-feature fusion and AI automatic point collection of the present invention; Figure 4 It is a schematic diagram of the measurement process steps after manual mouse point collection is converted into AI automatic point collection in an image measurement method based on multi-feature fusion and AI automatic point collection of the present invention; Figure 5 This is a schematic diagram of the architecture of an image measurement system based on multi-feature fusion and AI automatic point collection in the present invention; Figure 6 This is a schematic diagram of the architecture of an image measurement platform based on multi-feature fusion and AI automatic point sampling in the present invention. DETAILED DESCRIPTION

[0022] In order to better understand the purpose, technical solutions and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0023] The present invention may also be implemented or applied through other different specific examples, and the details in this specification may also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0024] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back...), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0025] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. Secondly, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] Preferably, the image measurement method based on multi-feature fusion and AI automatic point collection of the present invention is applied in one or more terminals or servers. The terminal is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0027] The terminal can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal can interact with the client through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

[0028] The present invention realizes an image measurement method, system and platform based on multi-feature fusion and AI automatic point sampling.

[0029] like Figure 1 , which is a flow chart of an image measurement method based on multi-feature fusion and AI automatic point collection provided by an embodiment of the present invention.

[0030] In this embodiment, the image measurement method based on multi-feature fusion and AI automatic point collection can be applied to a terminal with a display function or a fixed terminal. The terminal is not limited to a personal computer, a smart phone, a tablet computer, a desktop computer or an all-in-one computer equipped with a camera, etc.

[0031] The image measurement method based on multi-feature fusion and AI automatic point collection can also be applied to a hardware environment consisting of a terminal and a server connected to the terminal through a network. The network includes but is not limited to: a wide area network, a metropolitan area network or a local area network. The image measurement method based on multi-feature fusion and AI automatic point collection in the embodiment of the present invention can be executed by a server, can be executed by a terminal, or can be executed by both a server and a terminal.

[0032] For example, for an image measurement terminal that needs to perform image measurement based on multi-feature fusion and AI automatic point collection, the image measurement function based on multi-feature fusion and AI automatic point collection provided by the method of the present invention can be directly integrated on the terminal, or a client for implementing the method of the present invention can be installed. For another example, the method provided by the present invention can also be run on a server or other device in the form of a software development kit (SDK), and an interface for the image measurement function based on multi-feature fusion and AI automatic point collection is provided in the form of SDK. The terminal or other device can implement the image measurement function based on multi-feature fusion and AI automatic point collection through the provided interface. The present invention is further described below in conjunction with the accompanying drawings.

[0033] like Figure 1 As shown, the present invention provides an image measurement method based on multi-feature fusion and AI automatic point collection, and the method comprises the following steps: S1. Generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; S2. Based on the first data, in combination with the CSP backbone network structure and the adaptive anchor frame, respectively generate corresponding second data and third data; wherein the second data is image feature data; and the third data is anchor frame size data; S3, generating corresponding fourth data based on a multi-task learning framework, and marking and processing the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; S4. Construct a corresponding measurement element model according to the fifth data, and generate corresponding sixth data based on the measurement element model; wherein the sixth data is measurement data corresponding to the product to be measured.

[0034] The generating and acquiring first data corresponding to the workpiece to be measured and preprocessing the first data also includes: S11, generating and acquiring seventh data corresponding to the product to be measured and processed; wherein the seventh data is activation control data for starting to collect product image data; S12. Based on the seventh data, combined with data enhancement technology, the first data is preprocessed in real time to generate corresponding eighth data; wherein the preprocessing includes random scaling, cropping, flipping and translation; the eighth data is diversified training data corresponding to the image data.

[0035] The method of generating corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame also includes: S21, combining the CSP backbone network structure, sequentially splitting and merging the first data; S22. Create a data set corresponding to the first data, and analyze and process the data set to generate third data corresponding to the data set; wherein the data set includes image data of the product to be tested and corresponding annotation information data.

[0036] The method of generating corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame also includes: S23. Based on the CSP backbone network structure, calculate and generate second data corresponding to the first data; wherein the calculation formula is as follows: (1) In the formula, is the fused feature; are feature maps at different stages; S24, combined with the adaptive learning algorithm, dynamically adjust the anchor box size and loss function weight of the model; where the calculation formula of the anchor box size is as follows: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; It is the intersection and comparison.

[0037] The method of generating corresponding fourth data based on a multi-task learning framework, and marking and processing the second data according to the fourth data to generate corresponding fifth data, further includes: S31. Combined with the multi-task learning framework, the corresponding classification loss data, regression loss data and confidence loss data are calculated and generated respectively; S32: Based on the second data and in combination with the first data, calculate and generate corresponding fifth data; wherein the calculation formula is as follows: (4) In the formula, For automatic sampling point location; For location confidence level; For location With real frame The intersection and union ratio of The method of constructing a corresponding measurement element model according to the fifth data and generating corresponding sixth data based on the measurement element model further includes: S41. Construct at least one corresponding measurement element according to the fifth data; wherein the measurement element includes a line, an arc and a distance; S42. Combined with stochastic gradient descent or Adam optimizer, respectively train and optimize the target detection model or key point detection model based on convolutional neural network.

[0038] Specifically, in an embodiment of the present invention, an image measurement method based on multi-feature fusion and AI automatic point collection is provided, which realizes efficient and high-precision automatic measurement by combining AI automatic point collection technology with multi-feature fusion algorithm; the method comprises the following steps: the user opens the measurement program in the image measuring instrument software and selects the "multi-feature fusion AI automatic measurement" mode; places the product to be measured, starts the measurement program, and the software automatically collects product images; pre-processes the image by data enhancement technology, including random scaling, cropping, flipping and translation, to generate diversified training data; and uses the CSP (Cross Stage Partial) backbone network structure to extract image features, and the feature extraction formula is: (1) In the formula, is the fused feature; are feature maps at different stages; the anchor box size is determined by adaptive anchor box calculation, and the anchor box size calculation formula is: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; is the intersection-over-union ratio. Using the multi-task learning framework, the classification loss, regression loss and confidence loss are calculated, and the total loss function is: (3) In the formula, is the weight coefficient; is the classification loss; is the regression loss; is the confidence loss; According to the extracted features, the software automatically marks the measurement points in the image, and the calculation formula for the sampling point position is: (4) In the formula, For automatic sampling point location; For location confidence level; For location With real frame The intersection and union ratio of The area in the image where points are to be collected.

[0039] The software constructs other measurement elements, including lines, arcs, and distances, based on the location information of the automatically collected points; outputs measurement data and reports to complete the measurement task; and uses stochastic gradient descent (SGD) or Adam optimizer to train and optimize the model.

[0040] Among them, the CSP backbone network structure splits and fuses the feature map at different stages to enhance the propagation and reuse of features. The data enhancement technology includes at least one of random scaling, cropping, flipping and translation. The adaptive anchor frame calculation adaptively calculates the anchor frame size suitable for the data set by analyzing the data set. The multi-task learning framework includes classification loss, regression loss and confidence loss, and performs comprehensive supervised learning on different outputs of the model. The location information of the automatic point collection is recorded and saved in the database as reference data for subsequent measurements. The construction of other measurement elements includes at least one of lines, arcs and distances. The model reduces the number of parameters and calculations through lightweight design and improves the running speed. The optimizer includes stochastic gradient descent (SGD) and its variant Adam. The method is suitable for the size measurement of industrial products with complex surface features.

[0041] Example: Implementation of multi-feature fusion and AI automatic point collection For data collection and preprocessing, the user opens the measurement program in the image measuring instrument software and selects the "multi-feature fusion AI automatic measurement" mode; places the product to be measured (such as precision mechanical parts), starts the measurement program, and the software automatically collects product images; preprocesses the images through data enhancement technology, such as random scaling (scaling ratio range is 0.8~1.2), cropping (randomly cropping 80%~100% of the image area), flipping (horizontally or vertically flipping) and translating (randomly translating ±10% of the image range) to generate diverse training data.

[0042] Feature extraction and fusion, using the CSP (Cross Stage Partial) backbone network structure to extract image features. For example, if the input image size is 512×512 pixels, after passing through the CSP network, the output feature map size is 16×16×256, where 256 is the number of feature channels; the anchor frame size is determined by adaptive anchor frame calculation. For example, after analyzing 1000 samples in the data set, the appropriate anchor frame size is calculated to be [32, 64, 128] pixels.

[0043] AI automatically collects points and uses a multi-task learning framework to calculate classification loss, regression loss, and confidence loss. For example, the classification loss uses the cross entropy loss function, the regression loss uses the smooth L1 loss function, and the confidence loss uses the binary cross entropy loss function; based on the extracted features, the software automatically marks the measurement points in the image. For example, the edge points of the parts are marked in the image, and the calculation formula for the sampling point position is: (4) In the formula, For automatic sampling point location; For location confidence level; For location With real frame The intersection and union ratio of The area to be sampled in the image. Measurement and result output: The software constructs other measurement elements based on the position information of the automatically sampled points. For example, it constructs straight lines and arcs based on the marked edge points and calculates their lengths and radii; it outputs measurement data and reports to complete the measurement task.

[0044] Model optimization and update: Stochastic gradient descent (SGD) optimizer is used to train and optimize the model with an initial learning rate of 0.001 and a momentum parameter of 0.9. The model parameters and calculation amount are reduced through lightweight design. For example, the model parameters are compressed from 100MB to 10MB to improve the running speed.

[0045] The core of the solution of the present invention is to convert the manual point collection process into an automatic point collection process through multi-feature fusion and AI automatic point collection technology. Specifically, in multi-feature fusion, the multi-scale features in the image are extracted through the CSP backbone network, and the feature matching accuracy is improved by combining adaptive anchor frame calculation; in AI automatic point collection, the location of the measurement point is automatically determined through the multi-task learning framework and confidence calculation to reduce human operation errors; and for model optimization, the model's running speed and measurement efficiency are improved through lightweight design and optimizer selection.

[0046] In another embodiment of the solution of the present invention, a method for converting the elements of mouse point measurement from manual measurement to AI automatic measurement (i.e., a method for converting manual point measurement to automatic point measurement) in a two-dimensional image measurement device is provided; Figure 2-Figure 4 As shown in the figure, since manual work is replaced automatically, the measurement efficiency is greatly improved, and the measurement accuracy is also guaranteed through AI feature training (that is, the user's measurement efficiency is improved and the measurement accuracy is improved). Among them, the main technologies used in AI automatic point collection are: network structure design, using the CSP (Cross Stage Partial) backbone network structure to split and fuse feature maps at different stages, and enhance feature propagation and reuse.

[0047] Data enhancement technology uses a variety of data enhancement techniques, such as random scaling, cropping, flipping, translation, etc., to increase the diversity of data, enabling the model to learn more target features in different scenarios and improve the model's generalization ability and detection accuracy.

[0048] The training strategy is optimized and adaptive anchor box calculation is used. Before training, the anchor box size suitable for the data set is adaptively calculated through analysis of the data set, so that the model can better match the real bounding box of the target and improve the positioning accuracy.

[0049] The loss function design adopts a multi-part loss function including classification loss, regression loss and confidence loss to conduct comprehensive supervised learning on the different outputs of the model to ensure that the model can achieve high accuracy in classification and positioning.

[0050] Optimizer selection usually uses stochastic gradient descent (SGD) and its variants, such as Adam and other optimizers, and adjusts hyperparameters such as learning rate so that the model can quickly converge to the optimal solution, thereby ensuring detection accuracy.

[0051] Model optimization and improvement, under the premise of ensuring detection accuracy, the model is designed to be lightweight, the number of model parameters and the amount of calculation are reduced, and the running speed of the model is improved. It also helps to prevent overfitting and further improve detection accuracy.

[0052] To achieve the above objectives, the present invention also provides an image measurement system based on multi-feature fusion and AI automatic point sampling, such as Figure 5 As shown, the system specifically includes: A first data generating unit, used to generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; A second data generating unit is used to generate corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame, wherein the second data is image feature data; and the third data is anchor frame size data; A first data processing unit is used to generate corresponding fourth data based on a multi-task learning framework, and mark and process the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; The third data generating unit is used to construct a corresponding measurement element model according to the fifth data, and generate corresponding sixth data based on the measurement element model; wherein the sixth data is measurement data corresponding to the product to be measured.

[0053] The first data generating unit further includes: A first generating module is used to generate and obtain seventh data corresponding to the product to be measured and processed; wherein the seventh data is activation control data for starting to collect product image data; A first processing module is used to pre-process the first data in real time according to the seventh data in combination with data enhancement technology, and generate corresponding eighth data; wherein the pre-processing includes random scaling processing, cropping processing, flipping processing and translation processing; the eighth data is diversified training data corresponding to the image data; And / or, the second data generating unit further includes: A second processing module is used to sequentially split and merge the first data in combination with the CSP backbone network structure; A third processing module, used to create a data set corresponding to the first data, and analyze and process the data set to generate third data corresponding to the data set; wherein the data set includes image data of the product to be tested and corresponding annotation information data; And / or, the first data processing unit further includes: The second generation module is used to combine the multi-task learning framework to calculate and generate corresponding classification loss data, regression loss data and confidence loss data respectively; The first calculation module is used to calculate and generate corresponding fifth data based on the second data and in combination with the first data; wherein the calculation formula is as follows: (4) In the formula, For automatic sampling point location; For location confidence level; For location With real frame The intersection and union ratio of is the area range of the image to be sampled; and / or, the third data generating unit further includes: A first construction module, configured to construct at least one corresponding measurement element according to the fifth data; wherein the measurement element includes a line, an arc and a distance; The fourth processing module is used to combine the stochastic gradient descent or Adam optimizer to respectively train and optimize the target detection model or the key point detection model based on the convolutional neural network.

[0054] The second data generating unit further includes: The second calculation module is used to calculate and generate second data corresponding to the first data based on the CSP backbone network structure; wherein the calculation formula is as follows: (1) In the formula, is the fused feature; are feature maps at different stages; the fifth processing module is used to dynamically adjust the anchor box size and loss function weight of the model in combination with the adaptive learning algorithm; wherein the calculation formula of the anchor box size is as follows: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; It is the intersection and comparison.

[0055] In the system solution embodiment of the present invention, the method steps involved in the image measurement based on multi-feature fusion and AI automatic point collection have been described above in detail, that is, the functional modules in the system are used to implement the steps or sub-steps in the above method embodiment, which will not be repeated here.

[0056] To achieve the above objectives, the present invention also provides an image measurement platform based on multi-feature fusion and AI automatic point sampling, such as Figure 6As shown, it includes a processor, a memory, and an image measurement platform control program based on multi-feature fusion and AI automatic point collection; wherein, the image measurement platform control program based on multi-feature fusion and AI automatic point collection is executed by the processor, the image measurement platform control program based on multi-feature fusion and AI automatic point collection is stored in the memory, and the image measurement platform control program based on multi-feature fusion and AI automatic point collection implements the steps of the image measurement method based on multi-feature fusion and AI automatic point collection. For example: S1. Generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; S2. Based on the first data, in combination with the CSP backbone network structure and the adaptive anchor frame, respectively generate corresponding second data and third data; wherein the second data is image feature data; and the third data is anchor frame size data; S3, generating corresponding fourth data based on a multi-task learning framework, and marking and processing the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; S4. Construct a corresponding measurement element model according to the fifth data, and generate corresponding sixth data based on the measurement element model; wherein the sixth data is measurement data corresponding to the product to be measured.

[0057] The specific details of the steps have been explained above and will not be repeated here.

[0058] In the embodiment of the present invention, the built-in processor of the image measurement platform based on multi-feature fusion and AI automatic point collection can be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor uses various interfaces and lines to connect various components, and executes or executes programs or units stored in the memory, and calls data stored in the memory to perform various functions of image measurement based on multi-feature fusion and AI automatic point collection and process data; The memory is used to store program codes and various data. It is installed in the image measurement platform based on multi-feature fusion and AI automatic point collection, and realizes high-speed and automatic access to programs or data during operation.

[0059] The memory includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electronically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0060] The present invention generates and obtains first data corresponding to a product to be measured and processed through a method, and pre-processes the first data; wherein the first data is product image data; based on the first data, and in combination with a CSP backbone network structure and an adaptive anchor frame, corresponding second data and third data are generated respectively; wherein the second data is image feature data; the third data is anchor frame size data; corresponding fourth data is generated based on a multi-task learning framework, and according to the fourth data, the second data is marked and processed to generate corresponding fifth data; wherein the fourth data is loss function data; the fifth data is sampling point location information data; according to the fifth data, a corresponding measurement element model is constructed, and based on the measurement element model, corresponding sixth data is generated; wherein the sixth data is measurement data corresponding to the product to be measured, as well as a system and platform corresponding to the method, through AI automatic sampling technology combined with a multi-feature fusion algorithm, efficient and high-precision automatic measurement is achieved.

[0061] That is to say, the solution of the present invention replaces manual operation with automatic point sampling by AI, which significantly reduces the measurement time and is particularly suitable for large-scale measurement tasks; it utilizes multi-feature fusion technology to fully extract multi-scale features in the image, and combines adaptive anchor frame calculation and multi-task learning framework to ensure high-precision positioning of the measurement points; through data enhancement technology and CSP backbone network, the model's recognition ability for complex surface features is improved, which is suitable for the measurement of complex structures such as irregular contours and tiny holes; the automatic point sampling process is completely completed by the AI ​​model, avoiding errors introduced by human operation and ensuring the consistency of the measurement results; through lightweight design and optimizer selection, the model parameters and calculation amount are reduced, the running speed is improved, and it is suitable for real-time measurement scenarios.

[0062] In other words, the solution of the present invention improves measurement accuracy, enhances measurement adaptability, reduces human errors, and makes the model lightweight; that is, it solves the problems of low efficiency, unstable accuracy, and poor adaptability of traditional manual point measurement methods.

[0063] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An image measurement method based on multi-feature fusion and AI automatic point sampling, characterized in that: The method comprises the steps of: Generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; Based on the first data, in combination with the CSP backbone network structure and the adaptive anchor frame, corresponding second data and third data are generated respectively; wherein the second data is image feature data; and the third data is anchor frame size data; Generate corresponding fourth data based on a multi-task learning framework, and mark and process the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; According to the fifth data, a corresponding measurement element model is constructed, and based on the measurement element model, corresponding sixth data is generated; wherein the sixth data is measurement data corresponding to the product to be measured.

2. The image measurement method based on multi-feature fusion and AI automatic point sampling according to claim 1 is characterized in that: The generating and acquiring first data corresponding to the workpiece to be measured and preprocessing the first data also includes: Generate and obtain seventh data corresponding to the product to be measured and processed; wherein the seventh data is activation control data for starting to collect product image data; According to the seventh data, combined with data enhancement technology, the first data is preprocessed in real time to generate corresponding eighth data; wherein the preprocessing includes random scaling, cropping, flipping and translation; the eighth data is diversified training data corresponding to the image data.

3. The image measurement method based on multi-feature fusion and AI automatic point sampling according to claim 1 is characterized in that: The method of generating corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame also includes: Combined with the CSP backbone network structure, the first data is sequentially split and merged; A data set corresponding to the first data is created, and the data set is analyzed and processed to generate third data corresponding to the data set; wherein the data set includes image data of the product to be tested and corresponding annotation information data.

4. The image measurement method based on multi-feature fusion and AI automatic point sampling according to claim 1 or 3, characterized in that: The method of generating corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame also includes: Based on the CSP backbone network structure, second data corresponding to the first data is calculated and generated; wherein the calculation formula is as follows: (1) In the formula, is the fused feature; It is the feature map of different stages; Combined with the adaptive learning algorithm, the anchor box size and loss function weight of the model are dynamically adjusted; the calculation formula of the anchor box size is as follows: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; It is the intersection and comparison.

5. The image measurement method based on multi-feature fusion and AI automatic point sampling according to claim 1 is characterized in that: The method of generating corresponding fourth data based on a multi-task learning framework, and marking and processing the second data according to the fourth data to generate corresponding fifth data, further includes: Combined with the multi-task learning framework, the corresponding classification loss data, regression loss data and confidence loss data are calculated and generated respectively; Based on the second data and in combination with the first data, the corresponding fifth data is calculated and generated; wherein the calculation formula is as follows: (4) In the formula, For automatic sampling point location; For location Confidence level; For location With real box The intersection and union ratio of The area in the image where points are to be sampled.

6. The image measurement method based on multi-feature fusion and AI automatic point sampling according to claim 1 is characterized in that: The step of constructing a corresponding measurement element model according to the fifth data, and generating corresponding sixth data based on the measurement element model, further includes: According to the fifth data, construct at least one corresponding measurement element; wherein the measurement element includes a line, an arc and a distance; Combined with stochastic gradient descent or Adam optimizer, the convolutional neural network-based object detection model or key point detection model is trained and optimized respectively.

7. An image measurement system based on multi-feature fusion and AI automatic point sampling, characterized in that: The system is used to implement the image measurement method based on multi-feature fusion and AI automatic point collection as described in any one of claims 1 to 6, and the system includes: A first data generating unit, used to generate and obtain first data corresponding to the product to be measured and processed, and pre-process the first data; wherein the first data is product image data; A second data generating unit is used to generate corresponding second data and third data based on the first data and in combination with the CSP backbone network structure and the adaptive anchor frame, wherein the second data is image feature data; and the third data is anchor frame size data; A first data processing unit is used to generate corresponding fourth data based on a multi-task learning framework, and mark and process the second data according to the fourth data to generate corresponding fifth data; wherein the fourth data is loss function data; and the fifth data is sampling point location information data; The third data generating unit is used to construct a corresponding measurement element model according to the fifth data, and generate corresponding sixth data based on the measurement element model; wherein the sixth data is measurement data corresponding to the product to be measured.

8. The image measurement system based on multi-feature fusion and AI automatic point sampling according to claim 7 is characterized in that: The first data generating unit further includes: A first generating module is used to generate and obtain seventh data corresponding to the product to be measured and processed; wherein the seventh data is activation control data for starting to collect product image data; A first processing module is used to pre-process the first data in real time according to the seventh data in combination with data enhancement technology, and generate corresponding eighth data; wherein the pre-processing includes random scaling processing, cropping processing, flipping processing and translation processing; the eighth data is diversified training data corresponding to the image data; And / or, the second data generating unit further includes: A second processing module is used to sequentially split and merge the first data in combination with the CSP backbone network structure; A third processing module, used to create a data set corresponding to the first data, and analyze and process the data set to generate third data corresponding to the data set; wherein the data set includes image data of the product to be tested and corresponding annotation information data; And / or, the first data processing unit further includes: The second generation module is used to combine the multi-task learning framework to calculate and generate corresponding classification loss data, regression loss data and confidence loss data respectively; The first calculation module is used to calculate and generate corresponding fifth data based on the second data and in combination with the first data; wherein the calculation formula is as follows: (4) In the formula, For automatic sampling point location; For location Confidence level; For location With real box The intersection and union ratio of is the area range of the image to be sampled; and / or, the third data generating unit further includes: A first construction module, configured to construct at least one corresponding measurement element according to the fifth data; wherein the measurement element includes a line, an arc and a distance; The fourth processing module is used to combine the stochastic gradient descent or Adam optimizer to respectively train and optimize the target detection model or the key point detection model based on the convolutional neural network.

9. The image measurement system based on multi-feature fusion and AI automatic point sampling according to claim 7 or 8, characterized in that: The second data generating unit further includes: The second calculation module is used to calculate and generate second data corresponding to the first data based on the CSP backbone network structure; wherein the calculation formula is as follows: (1) In the formula, is the fused feature; are feature maps at different stages; the fifth processing module is used to dynamically adjust the anchor box size and loss function weight of the model in combination with the adaptive learning algorithm; wherein the calculation formula of the anchor box size is as follows: (2) In the formula, For the Anchor box size; is the prediction box; is the real frame; It is the intersection and comparison.

10. An image measurement platform based on multi-feature fusion and AI automatic point sampling, characterized in that: The invention comprises a processor, a memory and an image measurement platform control program based on multi-feature fusion and AI automatic point collection; wherein the image measurement platform control program based on multi-feature fusion and AI automatic point collection is executed by the processor, the image measurement platform control program based on multi-feature fusion and AI automatic point collection is stored in the memory, and the image measurement platform control program based on multi-feature fusion and AI automatic point collection implements the image measurement method based on multi-feature fusion and AI automatic point collection as described in any one of claims 1 to 6.

Images