Industrial image key point detection method, system and platform based on regression model

Abstract

The present invention discloses a method, system and platform for detecting key points of industrial images based on a regression model. The present invention obtains image data corresponding to the key points of industrial images through the method, and generates a feature map corresponding to the key points of industrial images in real time in combination with a feature extraction network; compresses the feature information of the feature map in real time through convolution processing, wherein the convolution is a 1×1 convolution; constructs a regression model, and generates detection data corresponding to the key points of industrial images in real time in combination with the regression model, as well as a system, platform and storage medium corresponding to the method; the global spatial information of the features can be retained, and the spatial generalization and stability of the key point prediction can be improved. In addition, the visibility function of each point can be added to the key point regression model.

Description

Technical Field

[0001] The present invention belongs to the technical field of image key point detection and processing, and in particular relates to an industrial image key point detection method, system and platform based on a regression model. Background Art

[0002] At present, for traditional key point detection algorithms, the detected key points have weak adaptability and cannot learn the features of different samples from a large amount of data, resulting in low accuracy of key point detection.

[0003] In addition, global pooling is basically used in the regression model of key points, but global pooling will lose the spatial information of the feature map, making it difficult for the model to distinguish the position information of the key points. Moreover, the general regression model of key points will output all types of points, regardless of whether a certain type of point exists in the image. Points that do not exist in the image will also be detected, and the scores are very high. It is not possible to filter out invisible points in the image through thresholds.

[0004] For the key point regression model of residual likelihood estimation, the distribution probability density of the model output joints is estimated through the flow model. Once a satisfactory prior distribution function is estimated, the loss function can be dynamically optimized to promote the regression training of the model. However, the output loss function value is negative when the model converges, which is not intuitive, and the convergence of the model training process cannot be judged by the loss function value.

[0005] Therefore, in view of the above technical problems and defects, it is urgent to design and develop an industrial image key point detection method, system and platform based on regression model. Summary of the invention

[0006] In order to overcome the shortcomings and difficulties of the above-mentioned prior art, the purpose of the present invention is to provide an industrial image key point detection method, system, platform and storage medium based on a regression model, which can retain the global spatial information of the features and improve the spatial generalization and stability of key point prediction. It can also realize the visibility function of adding each point to the key point regression model.

[0007] The first object of the present invention is to provide an industrial image key point detection method based on a regression model;

[0008] The second object of the present invention is to provide an industrial image key point detection system based on a regression model;

[0009] The third object of the present invention is to provide an industrial image key point detection platform based on a regression model;

[0010] A fourth object of the present invention is to provide a computer-readable storage medium;

[0011] The first object of the present invention is achieved in that the method comprises the following steps:

[0012] Acquire image data corresponding to the key points of industrial images, and combine with feature extraction network to generate feature maps corresponding to the key points of industrial images in real time;

[0013] Compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0014] Build a regression model and combine it to generate detection data corresponding to the key points of industrial images in real time.

[0015] Furthermore, the real-time compression of feature information of the feature map by convolution processing, wherein the convolution is a 1×1 convolution, further comprises:

[0016] Convolutionally compressing feature information of the feature map to generate a feature map having a number of channels corresponding to the feature map;

[0017] The feature map having the number of channels is normalized, a Relu activation function is activated, and features of the feature map are flattened.

[0018] Furthermore, the real-time compression of feature information of the feature map by convolution processing, wherein the convolution is a 1×1 convolution, further comprises:

[0019] Determine whether there are invisible key points. If so, remove the invisible key points based on the key point threshold; otherwise, proceed to the next step;

[0020] Based on the judgment result data, the key point information corresponding to the feature map is generated in real time.

[0021] Furthermore, the step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes:

[0022] Acquire key point information corresponding to the feature map in real time; and process the key point information by adding two to the output layer number of the fully connected layer;

[0023] Generate a key point with at least two flag data, wherein one flag indicates the key point exists and the second flag indicates the key point does not exist;

[0024] The flag values ​​are normalized using softmax and the indices corresponding to the key points are generated.

[0025] Furthermore, the step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes:

[0026] Generate position data corresponding to key points, as well as data information on whether key points exist or not.

[0027] Furthermore, the step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes:

[0028] Combined with the mean square error loss function data, the square sum of pixel differences between the model prediction value and the true value is generated in real time;

[0029] According to the loss function value data, the convergence state of the regression model during the training process is judged.

[0030] The second object of the present invention is achieved in that the system comprises:

[0031] An acquisition generation unit is used to acquire image data corresponding to the key points of the industrial image, and generate a feature map corresponding to the key points of the industrial image in real time in combination with a feature extraction network;

[0032] A convolution processing unit, used for compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0033] The detection data generation unit is used to build a regression model and generate detection data corresponding to key points of the industrial image in real time in combination with the regression model.

[0034] Furthermore, the convolution processing unit further includes:

[0035] A first generating module, configured to convolute and compress feature information of the feature map to generate a feature map having a number of channels corresponding to the feature map;

[0036] A first processing module is used to normalize the feature map having the number of channels, activate the Relu activation function, and flatten the features of the feature map;

[0037] And / or, the convolution processing unit further includes:

[0038] The first determination module is used to determine whether there are invisible key points. If so, the invisible key points are removed in combination with the key point threshold; otherwise, the key point information corresponding to the feature map is generated in real time according to the determination result data;

[0039] The second generation module is used to generate key point information corresponding to the feature map in real time according to the determination result data;

[0040] The detection data generating unit further includes:

[0041] The second processing module is used to obtain the key point information corresponding to the feature map in real time; and to process the key point information by adding two to the output layer number of the fully connected layer;

[0042] A third generation module is used to generate a key point with at least two flag data, wherein one flag indicates that the key point exists and the second flag indicates that the key point does not exist;

[0043] The fourth generation module is used to perform softmax normalization processing on the value of the flag bit and generate an index corresponding to the key point;

[0044] And / or, the detection data generating unit further includes:

[0045] A fifth generating module, used to generate position data corresponding to the key point, and data information on whether the key point exists;

[0046] And / or, the detection data generating unit further includes:

[0047] The sixth generation module is used to generate the square sum of pixel differences between the model prediction value and the true value in real time by combining the mean square error loss function data;

[0048] The second determination module is used to determine the convergence state of the regression model during the training process according to the loss function value data.

[0049] The third object of the present invention is achieved as follows: it includes a processor, a memory and an industrial image key point detection platform control program based on a regression model; wherein the industrial image key point detection platform control program based on a regression model is executed by the processor, the industrial image key point detection platform control program based on a regression model is stored in the memory, and the industrial image key point detection platform control program based on a regression model implements the industrial image key point detection method based on a regression model.

[0050] The fourth object of the present invention is achieved in this way: the computer-readable storage medium stores an industrial image key point detection platform control program based on a regression model, and the industrial image key point detection platform control program based on a regression model implements the industrial image key point detection method based on a regression model.

[0051] The present invention obtains image data corresponding to the key points of industrial images through a method, and generates a feature map corresponding to the key points of industrial images in real time in combination with a feature extraction network; compresses the feature information of the feature map in real time through convolution processing, wherein the convolution is a 1×1 convolution; constructs a regression model, and generates detection data corresponding to the key points of industrial images in real time in combination with the regression model, as well as a system, platform and storage medium corresponding to the method; the global spatial information of the features can be retained, and the spatial generalization and stability of the key point prediction can be improved. Moreover, the visibility function of each point can be added to the key point regression model.

[0052] In other words, the industrial image key point detection algorithm based on the regression model can allow the equipment to automatically detect various types of key points in industrial images, and the average pixel error of each type of point can be controlled within one pixel. It has higher accuracy than traditional detection algorithms, and eliminates the influence of subjective factors on defect judgment caused by individual differences, physical conditions, etc., improving the safety, reliability and efficiency of detection. Compared with the common key point regression model, it has the function of predicting point visibility, which greatly reduces the average pass rate of the common key point regression model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 It is a schematic diagram of the network structure of a common key point regression model in the prior art;

[0055] Figure 2 A schematic diagram of the network structure of the FCNeck key point regression model of an industrial image key point detection method based on a regression model of the present invention;

[0056] Figure 3 The present invention is a flowchart of a method for detecting key points of industrial images based on a regression model;

[0057] Figure 4 This is a schematic diagram of the architecture of an industrial image key point detection system based on a regression model of the present invention;

[0058] Figure 5 This is a schematic diagram of an industrial image key point detection platform architecture based on a regression model of the present invention;

[0059] Figure 6A schematic diagram of a computer-readable storage medium architecture in an embodiment of the present invention;

[0060] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0061] In order to better understand the purpose, technical solutions and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification.

[0062] The present invention may also be implemented or applied through other different specific examples, and the details in this specification may also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0063] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0064] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. Secondly, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0065] Preferably, the industrial image key point detection method based on regression model of the present invention is applied in one or more terminals or servers. The terminal is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), digital signal processors (DSP), embedded devices, etc.

[0066] The terminal can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal can interact with the client through a keyboard, a mouse, a remote control, a touch pad, or a voice control device.

[0067] The present invention is to realize a method, system, platform and storage medium for detecting key points of industrial images based on a regression model.

[0068] like Figure 3 , which is a flow chart of an industrial image key point detection method based on a regression model provided in an embodiment of the present invention.

[0069] In this embodiment, the industrial image key point detection method based on the regression model can be applied to a terminal with a display function or a fixed terminal. The terminal is not limited to a personal computer, a smart phone, a tablet computer, a desktop computer or an all-in-one computer equipped with a camera, etc.

[0070] The industrial image key point detection method based on the regression model can also be applied to a hardware environment consisting of a terminal and a server connected to the terminal through a network. The network includes but is not limited to: a wide area network, a metropolitan area network or a local area network. The industrial image key point detection method based on the regression model of the embodiment of the present invention can be executed by a server, or by a terminal, or by both a server and a terminal.

[0071] For example, for a terminal that needs to perform industrial image key point detection based on a regression model, the industrial image key point detection function based on a regression model provided by the method of the present invention can be directly integrated on the terminal, or a client for implementing the method of the present invention can be installed. For another example, the method provided by the present invention can also be run on a server or other device in the form of a software development kit (SDK), and an interface for the industrial image key point detection function based on a regression model is provided in the form of SDK. The terminal or other device can implement the industrial image key point detection function based on the regression model through the provided interface. The present invention is further described below in conjunction with the accompanying drawings.

[0072] like Figure 1-Figure 3 As shown, the present invention provides an industrial image key point detection method based on a regression model, and the method comprises the following steps:

[0073] S1, obtain image data corresponding to the key points of the industrial image, and combine with the feature extraction network to generate feature maps corresponding to the key points of the industrial image in real time;

[0074] S2. compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0075] S3. Build a regression model and combine it to generate detection data corresponding to the key points of the industrial image in real time.

[0076] The real-time compression of feature information of the feature map by convolution processing, wherein the convolution is a 1×1 convolution, further comprises:

[0077] S21, convolutionally compressing feature information of the feature map to generate a feature map having a number of channels corresponding to the feature map;

[0078] S22, normalize the feature map having the number of channels, activate the Relu activation function, and flatten the features of the feature map.

[0079] The real-time compression of feature information of the feature map by convolution processing, wherein the convolution is a 1×1 convolution, further comprises:

[0080] S23, determine whether there are invisible key points, if so, remove the invisible key points in combination with the key point threshold; otherwise, execute the next step;

[0081] S24. Generate key point information corresponding to the feature map in real time based on the determination result data.

[0082] The step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes:

[0083] S31, acquiring key point information corresponding to the feature map in real time; and processing the key point information by adding two to the output layer number of the fully connected layer;

[0084] S32, generating a key point with at least two flag data, wherein one flag indicates that the key point exists, and the second flag indicates that the key point does not exist;

[0085] S33, perform softmax normalization processing on the value of the flag bit, and generate an index corresponding to the key point.

[0086] The step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes:

[0087] S34: Generate position data corresponding to the key point, as well as data information on whether the key point exists or not.

[0088] The step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes:

[0089] S35, combining the mean square error loss function data, generating the square sum of pixel differences between the model prediction value and the true value in real time;

[0090] S36. According to the loss function value data, determine the convergence state of the regression model during the training process.

[0091] Specifically, in an embodiment of the present invention, the technical background of using AI for key point detection in industry can be traced back to the development of computer vision and machine learning. Compared with traditional key point detection algorithms, AI key point detection has strong adaptability, and the AI ​​model can learn the characteristics of different samples from a large amount of data, and has stronger adaptability. The AI ​​model can be trained with a deep learning algorithm to obtain a more accurate model, thereby improving the accuracy of key point detection. The AI ​​model can be reused and optimized in different industrial application scenarios through technologies such as transfer learning, reducing development and maintenance costs. The AI ​​model can continuously optimize the model by continuously adding and updating data to adapt to new industrial application scenarios, and has stronger scalability.

[0092] That is, common key point regression models such as Figure 1 This solution changes the global pooling in the common key point regression model into a bottleneck structure with an additional design, named FCNeck. The corresponding network structure is as follows Figure 2 As shown. FCNeck first uses 1×1 convolution to significantly reduce the number of channels in the feature map, because the input and output channels of the convolution kernel used are 2048 and 32 respectively. Generally speaking, the number of fully connected hidden layers is affected by the number of fully connected input and output layers. If the number of input layer nodes is too large, more hidden layers are required. Therefore, using this convolution to compress feature information can also ensure that the hidden layer of the subsequent fully connected structure does not have too many parameters. Next, FCNeck will perform BN normalization and Relu activation function activation on the feature map of the reduced channel, and then flatten the features instead of the global pooling operation, avoiding the loss of feature map spatial information in the global pooling operation and retaining the spatial information of the feature map.

[0093] like Figure 1 The output of the common key point regression model shown in the figure outputs the location information of all types of key points, without considering the existence of the key points. The scores of non-existent points are still very high, and the invisible points cannot be removed by the threshold. The output head of the model of this scheme uses a fully connected layer to obtain the information of each key point, and the number of output layers of the fully connected layer that obtains the key point information is doubled, so that two more flags are added to the output of each key point, one flag indicates the existence of the key point, and the second indicates the non-existence. The values ​​of the two flags are softmax normalized and the index corresponding to the maximum value is found to indicate whether the key point exists. The output part of these two flags uses the cross entropy loss function, and the model structure is as follows Figure 2 As shown in the figure, the model can not only output the detailed location of the key points but also output the information of whether the key points exist or not.

[0094] The key point regression model of residual likelihood estimation estimates the distribution probability density of the model output joints through the flow model. However, the loss function value of its output is negative when the model converges, which is not intuitive, and the convergence of the model training process cannot be judged by the loss function value. This solution adds a mean square error loss function to the loss function to make the loss function value more intuitive, because the output of the mean square error loss function is the sum of the squares of the pixel differences between the model prediction value and the true value. When the value of this part of the loss function on the test set is close to 0, the model is basically converged, and the convergence of the model during the training process can be judged by the loss function value. At the same time, it can make the training process more stable. The total loss function of the model in this solution is the sum of the loss part of the residual likelihood estimation, the mean square error loss function part, and the key point visibility loss function part.

[0095] In other words, the main protection of this solution is: the key point regression model removes the common global pooling operation in the model and replaces it with other structures without global pooling, such as the bottleneck structure FCNeck in this solution, to retain the global spatial information of the features and improve the spatial generalization and stability of key point prediction. The key point regression model adds the visibility function of each point. The loss function implementation method of the key point regression model in this solution.

[0096] This scheme proposes a bottleneck structure of a key point model to convert the features extracted by the regression model feature extraction network into a fully connected input while retaining the spatial information of the feature map.

[0097] This solution adds a visibility loss function for key points. It outputs the visibility of each type of point while outputting its location information. This solves the problem that the model outputs high scores for points that do not exist in the image but are in the detectable categories of the model.

[0098] This solution adds a mean square error loss function to the loss function, making the loss function value more intuitive. The convergence of the model during training can be judged by the loss function value. At the same time, it can make the training process more stable.

[0099] After the image is preprocessed by the dataset, it is sent to the key point model. The feature extraction network of the model extracts features from the preprocessed image, and then sends the features to our FCNeck structure to retain the global information of the image as much as possible. The output features of the FCNeck structure are then fully connected to obtain the accurate position and visibility of each key point.

[0100] To achieve the above object, the present invention also provides an industrial image key point detection system based on a regression model, such as Figure 4 As shown, the system specifically includes:

[0101] An acquisition generation unit is used to acquire image data corresponding to the key points of the industrial image, and generate a feature map corresponding to the key points of the industrial image in real time in combination with a feature extraction network;

[0102] A convolution processing unit, used for compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0103] The detection data generation unit is used to build a regression model and generate detection data corresponding to key points of the industrial image in real time in combination with the regression model.

[0104] The convolution processing unit further includes:

[0105] A first generating module, configured to convolute and compress feature information of the feature map to generate a feature map having a number of channels corresponding to the feature map;

[0106] A first processing module is used to normalize the feature map having the number of channels, activate the Relu activation function, and flatten the features of the feature map;

[0107] And / or, the convolution processing unit further includes:

[0108] The first determination module is used to determine whether there are invisible key points. If so, the invisible key points are removed in combination with the key point threshold; otherwise, the key point information corresponding to the feature map is generated in real time according to the determination result data;

[0109] The second generation module is used to generate key point information corresponding to the feature map in real time according to the determination result data;

[0110] The detection data generating unit further includes:

[0111] The second processing module is used to obtain the key point information corresponding to the feature map in real time; and to process the key point information by adding two to the output layer number of the fully connected layer;

[0112] A third generation module is used to generate a key point with at least two flag data, wherein one flag indicates that the key point exists and the second flag indicates that the key point does not exist;

[0113] The fourth generation module is used to perform softmax normalization processing on the value of the flag bit and generate an index corresponding to the key point;

[0114] And / or, the detection data generating unit further includes:

[0115] A fifth generating module, used to generate position data corresponding to the key point, and data information on whether the key point exists;

[0116] And / or, the detection data generating unit further includes:

[0117] The sixth generation module is used to generate the square sum of pixel differences between the model prediction value and the true value in real time by combining the mean square error loss function data;

[0118] The second determination module is used to determine the convergence state of the regression model during the training process according to the loss function value data.

[0119] In the system solution embodiment of the present invention, the method steps involved in the industrial image key point detection based on the regression model have been described above in detail. That is to say, the functional modules in the system are used to implement the steps or sub-steps in the above method embodiment, which will not be repeated here.

[0120] To achieve the above objectives, the present invention also provides an industrial image key point detection platform based on a regression model, such as Figure 5 As shown, it includes a processor, a memory and a control program of an industrial image key point detection platform based on a regression model;

[0121] The processor executes the industrial image key point detection platform control program based on the regression model, the industrial image key point detection platform control program based on the regression model is stored in the memory, and the industrial image key point detection platform control program based on the regression model implements the steps of the industrial image key point detection method based on the regression model. For example:

[0122] S1, obtain image data corresponding to the key points of the industrial image, and combine with the feature extraction network to generate feature maps corresponding to the key points of the industrial image in real time;

[0123] S2. compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0124] S3. Build a regression model and combine it to generate detection data corresponding to the key points of the industrial image in real time.

[0125] The specific details of the steps have been explained above and will not be repeated here.

[0126] In the embodiment of the present invention, the built-in processor of the industrial image key point detection platform based on the regression model can be composed of an integrated circuit, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, and various control chips. The processor uses various interfaces and lines to connect various components, and executes or executes programs or units stored in the memory, and calls data stored in the memory to perform various functions and process data for industrial image key point detection based on the regression model;

[0127] The memory is used to store program codes and various data. It is installed in the industrial image key point detection platform based on the regression model and realizes high-speed and automatic access to programs or data during operation.

[0128] The memory includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electronically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0129] To achieve the above object, the present invention also provides a computer readable storage medium, such as Figure 6 As shown, the computer-readable storage medium stores a control program for an industrial image key point detection platform based on a regression model, and the control program for an industrial image key point detection platform based on a regression model implements the steps of the industrial image key point detection method based on a regression model, for example:

[0130] S1, obtain image data corresponding to the key points of the industrial image, and combine with the feature extraction network to generate feature maps corresponding to the key points of the industrial image in real time;

[0131] S2. compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0132] S3. Build a regression model and combine it to generate detection data corresponding to the key points of the industrial image in real time.

[0133] The specific details of the steps have been explained above and will not be repeated here.

[0134] In the description of the embodiments of the present invention, it should be noted that any process or method description in the flowchart or otherwise described herein may be understood as representing a module, fragment or portion of a code comprising one or more executable instructions for implementing steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations, in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).

[0136] In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0137] In an embodiment of the present invention, in order to achieve the above-mentioned purpose, the present invention further provides a chip system, wherein the chip system includes at least one processor, and when the program instructions are executed in the at least one processor, the chip system executes the steps of the industrial image key point detection method based on the regression model, for example:

[0138] S1, obtain image data corresponding to the key points of the industrial image, and combine with the feature extraction network to generate feature maps corresponding to the key points of the industrial image in real time;

[0139] S2. compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution;

[0140] S3. Build a regression model and combine it to generate detection data corresponding to the key points of the industrial image in real time.

[0141] The specific details of the steps have been explained above and will not be repeated here.

[0142] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0143] The present invention obtains image data corresponding to the key points of industrial images through a method, and generates a feature map corresponding to the key points of industrial images in real time in combination with a feature extraction network; compresses the feature information of the feature map in real time through convolution processing, wherein the convolution is a 1×1 convolution; constructs a regression model, and generates detection data corresponding to the key points of industrial images in real time in combination with the regression model, as well as a system, platform and storage medium corresponding to the method; the global spatial information of the features can be retained, and the spatial generalization and stability of the key point prediction can be improved. Moreover, the visibility function of each point can be added to the key point regression model.

[0144] In other words, the industrial image key point detection algorithm based on the regression model can allow the equipment to automatically detect various types of key points in industrial images, and the average pixel error of each type of point can be controlled within one pixel. It has higher accuracy than traditional detection algorithms, and eliminates the influence of subjective factors on defect judgment caused by individual differences, physical conditions, etc., improving the safety, reliability and efficiency of detection. Compared with the common key point regression model, it has the function of predicting point visibility, which greatly reduces the average pass rate of the common key point regression model.

[0145] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting key points of industrial images based on a regression model, characterized in that: The method comprises the following steps: Acquire image data corresponding to the key points of industrial images, and combine with feature extraction network to generate feature maps corresponding to the key points of industrial images in real time; Compressing the feature information of the feature map in real time by convolution processing; including: compressing the feature information of the feature map by convolution, and generating a feature map with a number of channels corresponding to the feature map; normalizing the feature map with a number of channels, activating a Relu activation function, and flattening the features of the feature map; wherein the convolution is a 1×1 convolution; Construct a regression model Construct a regression model, and combine the regression model to generate detection data corresponding to the key points of the industrial image in real time; including: obtaining the key point information corresponding to the feature map in real time; and processing the key point information by adding two to the output layer number of the fully connected layer; generating a key point with at least two flag data, wherein one flag indicates the existence of the key point and the second indicates the absence of the key point; performing softmax normalization on the flag value and generating an index corresponding to the key point.

2. The industrial image key point detection method based on regression model according to claim 1, characterized in that: The real-time compression of feature information of the feature map by convolution processing, wherein the convolution is a 1×1 convolution, further comprises: Determine whether there are invisible key points. If so, remove the invisible key points based on the key point threshold; otherwise, proceed to the next step; Based on the judgment result data, the key point information corresponding to the feature map is generated in real time.

3. The industrial image key point detection method based on regression model according to claim 1 is characterized in that: The step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes: Generate position data corresponding to key points, as well as data information on whether key points exist or not.

4. The industrial image key point detection method based on regression model according to claim 1 is characterized in that: The step of constructing a regression model and combining the regression model to generate detection data corresponding to key points of the industrial image in real time also includes: Combined with the mean square error loss function data, the square sum of pixel differences between the model prediction value and the true value is generated in real time; According to the loss function value data, the convergence state of the regression model during the training process is judged.

5. A system for detecting key points of industrial images based on a regression model for detecting key points of industrial images based on a regression model according to any one of claims 1 to 4, characterized in that: The system comprises: An acquisition generation unit is used to acquire image data corresponding to the key points of the industrial image, and generate a feature map corresponding to the key points of the industrial image in real time in combination with a feature extraction network; A convolution processing unit, used for compressing the feature information of the feature map in real time by convolution processing, wherein the convolution is a 1×1 convolution; The detection data generation unit is used to build a regression model and generate detection data corresponding to key points of the industrial image in real time in combination with the regression model.

6. The industrial image key point detection system based on regression model according to claim 5, characterized in that: The convolution processing unit further includes: A first generating module, configured to convolute and compress feature information of the feature map to generate a feature map having a number of channels corresponding to the feature map; A first processing module is used to normalize the feature map having the number of channels, activate the Relu activation function, and flatten the features of the feature map; And / or, the convolution processing unit further includes: The first determination module is used to determine whether there are invisible key points. If so, the invisible key points are removed in combination with the key point threshold; otherwise, the key point information corresponding to the feature map is generated in real time according to the determination result data; The second generation module is used to generate key point information corresponding to the feature map in real time according to the determination result data; The detection data generating unit further includes: The second processing module is used to obtain the key point information corresponding to the feature map in real time; and to process the key point information by adding two to the output layer number of the fully connected layer; A third generation module is used to generate a key point with at least two flag data, wherein one flag indicates that the key point exists and the second flag indicates that the key point does not exist; The fourth generation module is used to perform softmax normalization processing on the value of the flag bit and generate an index corresponding to the key point; And / or, the detection data generating unit further includes: A fifth generating module, used to generate position data corresponding to the key point, and data information on whether the key point exists; And / or, the detection data generating unit further includes: The sixth generation module is used to generate the square sum of pixel differences between the model prediction value and the true value in real time by combining the mean square error loss function data; The second determination module is used to determine the convergence state of the regression model during the training process according to the loss function value data.

7. An industrial image key point detection platform based on regression model, characterized in that: It includes a processor, a memory and a control program of an industrial image key point detection platform based on a regression model; The processor executes the industrial image key point detection platform control program based on the regression model, and the industrial image key point detection platform control program based on the regression model is stored in the memory. The industrial image key point detection platform control program based on the regression model implements the industrial image key point detection method based on the regression model as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a regression model-based industrial image key point detection platform control program, and the regression model-based industrial image key point detection platform control program implements the regression model-based industrial image key point detection method as described in any one of claims 1 to 4.

Images