Visual inspection model construction method and system, readable storage medium and computer
By building a lightweight RT-DETR detection algorithm model and distributed training method, the data island problem is solved, and efficient visual detection model construction is realized, which improves communication efficiency and reduces computing pressure.
Patent Information
- Application Number
- CN202510597465.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-08-15
AI Technical Summary
In the context of data privacy protection, the "data silos" problem formed when there is no data sharing among data holders makes it difficult to effectively implement the centralized training method of deep learning object detection technology.
A lightweight RT-DETR detection algorithm model is built, a training sample set is formed through image data processing and annotation, and a feature extraction is used to extract lightweight GhostNet, and a distributed training method such as a federated averaging algorithm is used to aggregate the model, break the data islands, and build a visual detection model.
It realizes that on the premise of meeting privacy protection, improve communication efficiency, reduce communication overhead, alleviate computing pressure, and realize data-free interactive collaborative detection.
Smart Images
Figure CN120495840A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual detection technology, and in particular to a method, system, readable storage medium and computer for constructing a visual detection model. Background Art
[0002] With the continuous deepening of machine vision research, deep learning target detection technology has been widely used in various fields.
[0003] Image processing has become a research hotspot. However, centralized training methods based on deep learning require massive amounts of data. Collecting large amounts of data becomes particularly challenging given the growing need for data privacy protection. When data holders lack data sharing, "data silos" form. Distributed machine learning allows participants to conduct data analysis and machine learning modeling while ensuring privacy protection, offering a new approach to addressing these "data silos" and privacy issues. Summary of the Invention
[0004] Based on this, the purpose of the present invention is to provide a visual detection model construction method, system, readable storage medium and computer to at least solve the deficiencies in the above-mentioned technology.
[0005] The present invention proposes a method for constructing a visual detection model, comprising: Acquiring image data captured by an image acquisition device, and annotating the image data to form an image data set; Performing image data processing on the image dataset to increase sample diversity in the image dataset to obtain a corresponding training sample set; Constructing a lightweight RT-DETR detection algorithm model, and inputting the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters; Multiple clients are aggregated based on the local model parameters, and a visual detection model is constructed using the global model weights obtained by aggregation, and machine vision detection is implemented using the visual detection model.
[0006] Furthermore, the steps of constructing a lightweight RT-DETR detection algorithm model include: Define the backbone network of the model and use lightweight GhostNet instead of the backbone network ResNet to reduce the initial complexity of the RT-DETR model; Based on the RT-DETR model, the channel pruning method is used to further lightweight it to obtain a lightweight RT-DETR detection algorithm model.
[0007] Furthermore, the step of inputting the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters includes: The backbone network of the lightweight RT-DETR detection algorithm model is used to extract features from the training sample set: ; in, * is the convolution operation, b is the bias term, , n is the number of channels output feature map, is the convolution filter of the current layer. are the height and width of the output data respectively, k×k Convolution filters f The kernel size; Through Y Each feature in is subjected to a linear operation to obtain the required number of n feature maps, and s Ghost feature maps are generated according to the following function:
[0008] Where, y' i Indicates the first i feature maps, Φ i,j is a linear transformation function, expressed as a sequence i The feature map of j A linear transformation; The GhostNet is used to perform data processing on the training sample set to obtain corresponding local model parameters. The GhostNet is used to perform data processing on the training sample set to obtain corresponding local model parameters.
[0009] Furthermore, the steps of aggregating multiple clients based on the local model parameters and constructing a visual detection model using the aggregated global model weights include: Perform multiple rounds of gradient descent on each client, and introduce the federated averaging algorithm to perform distributed training on each client; A global model aggregation is performed on each of the clients, and a visual detection model is constructed using the obtained global model weights.
[0010] Furthermore, the calculation formula for the global model aggregation is: ; in, For the client The local model parameters of the local iteration multiple times, is the gradient of the local model, is the learning rate.
[0011] The present invention also proposes a visual detection model construction system, comprising: A data acquisition module, configured to acquire image data acquired by an image acquisition device and annotate the image data to form an image data set; a data processing module, configured to perform image data processing on the image dataset to increase sample diversity in the image dataset and obtain a corresponding training sample set; A model training module is used to construct a lightweight RT-DETR detection algorithm model and input the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters; A global aggregation module is used to aggregate multiple clients based on the local model parameters, and use the global model weights obtained by aggregation to build a visual detection model, and use the visual detection model to implement machine vision detection.
[0012] Furthermore, the model training module includes: The network definition unit is used to define the backbone network of the model and use the lightweight GhostNet instead of the backbone network ResNet to reduce the initial complexity of the RT-DETR model; The model building unit is used to further lightweight the RT-DETR model by adopting a channel pruning method to obtain a lightweight RT-DETR detection algorithm model.
[0013] Furthermore, the model training module also includes: A feature extraction unit is used to extract features from the training sample set using the backbone network of the lightweight RT-DETR detection algorithm model: ; in * is the convolution operation, b is the bias term, , n is the number of channels output feature map, is the convolution filter of the current layer. are the height and width of the output data respectively, k×k Convolution filters f The kernel size; Linear processing unit, used to Y Each feature in is subjected to a linear operation to obtain the required number of n feature maps, and s Ghost feature maps are generated according to the following function:
[0014] Where, y' i Indicates the first i feature maps, Φ i,j is a linear transformation function, expressed as a sequence i The feature map of j A linear transformation; The model training unit is used to use the GhostNet to perform data processing on the training sample set to obtain corresponding local model parameters.
[0015] Furthermore, the global aggregation module includes: A distributed training unit, configured to perform multiple rounds of gradient descent on each client and introduce a federated averaging algorithm to perform distributed training on each of the clients; The global aggregation unit is used to perform global model aggregation on each of the clients and construct a visual detection model using the obtained global model weights.
[0016] The present invention also provides a readable storage medium having a computer program stored thereon, which implements the above-mentioned visual detection model construction method when executed by a processor.
[0017] The present invention also proposes a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned visual detection model construction method when executing the computer program.
[0018] The visual detection model construction method, system, readable storage medium and computer in the present invention annotate and process the image data collected by the image acquisition device to obtain a corresponding training sample set, use the constructed lightweight RT-DETR detection algorithm model to perform model training on the training sample set to obtain local model parameters, aggregate multiple clients based on the local model parameters to construct a visual detection model, and use the visual detection model to realize machine vision detection. While breaking the barriers of data islands and realizing collaborative detection without data interaction, it greatly improves communication efficiency, reduces communication overhead, and alleviates computing pressure. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 Flowchart of the method for constructing a visual detection model in the first embodiment of the present invention; Figure 2 A structural block diagram of a visual detection model building system in a second embodiment of the present invention; Figure 3 FIG. 4 is a structural block diagram of a computer in a third embodiment of the present invention.
[0020] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0021] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.
[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0023] Example 1 See also Figure 1 , which shows a method for constructing a visual detection model in a first embodiment of the present invention, and specifically includes steps S101 to S104: S101, acquiring image data captured by an image acquisition device, and annotating the image data to form an image data set; The visual detection model construction method provided in this embodiment is a visual detection model construction method based on distributed training, which is applied to the cloud-edge server, wherein the edge server receives the global model weights sent from the cloud and the image data uploaded by the drone in real time for local model training, and the cloud receives the local model weights of each edge client for aggregation, and sends the aggregated new round of global weights to each client.
[0024] In a specific implementation, image data collected by an image acquisition device is obtained, wherein the image acquisition device includes but is not limited to a device with image acquisition function such as a drone, and the drone uploads the collected image data to an edge server, annotates the data and creates an image dataset; S102, performing image data processing on the image dataset to increase sample diversity in the image dataset to obtain a corresponding training sample set; In the specific implementation, the above-mentioned image dataset is subjected to operations such as flipping, cropping, and translation to increase sample diversity and divide the dataset into training and test sets.
[0025] S103, constructing a lightweight RT-DETR detection algorithm model, and inputting the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters; Furthermore, the step S103 specifically includes steps S1031 and S1032: S1031, define the backbone network of the model and use lightweight GhostNet instead of the backbone network ResNet to reduce the initial complexity of the RT-DETR model; S1032: Based on the RT-DETR model, further lightweight it by using a channel pruning method to obtain a lightweight RT-DETR detection algorithm model.
[0026] In the specific implementation, the backbone network of the lightweight RT-DETR detection algorithm model in this embodiment uses the GhostNet convolutional neural network to extract features from the image, which is specifically defined as follows:
[0027] in * is the convolution operation, b is the bias term, , n is the number of channels output feature map, is the convolution filter of the current layer. are the height and width of the output data respectively, k×k Convolution filters f The kernel size.
[0028] Through Y Apply a series of concise linear operations to each feature in to obtain the required number of n feature maps, generating s Ghost feature maps according to the following function:
[0029] Where, y' i Indicates the first i feature maps, Φ i,j is a linear transformation function, expressed as a sequence i The feature map of j A linear transformation.
[0030] The model training unit uses the GhostNet to perform data processing on the training sample set to obtain corresponding local model parameters.
[0031] S104, aggregating multiple clients based on the local model parameters, and constructing a visual detection model using the aggregated global model weights, and implementing machine vision detection using the visual detection model.
[0032] Furthermore, the step S104 specifically includes steps S1041 and S1042: S1041, performing multiple rounds of gradient descent on each client, and introducing a federated averaging algorithm to perform distributed training on each client; S1042: Perform global model aggregation on each of the clients, and use the obtained global model weights to construct a visual detection model.
[0033] In the specific implementation, the cloud server introduces the Federated Averaging Algorithm (FedAvg) to perform distributed training on each participating client. First, multiple rounds of gradient descent are performed on each client. The formula is as follows: ; Where, Before the iteration The parameters of the model, are the model parameters before iteration, Indicates the The amount of data owned by each client, Indicates the total data volume of all clients.
[0034] Then, the FedAvg algorithm is introduced on the cloud server to perform distributed training on each participating client, and multiple rounds of gradient descent are performed on each client to obtain the global model aggregation calculation formula as follows: ; in, For the client The local model parameters of the local iteration multiple times, is the gradient of the local model, is the learning rate.
[0035] Furthermore, a global model aggregation is performed on each of the clients, and a visual detection model is constructed using the obtained global model weights.
[0036] In summary, the visual detection model construction method in the above embodiment of the present invention labels and processes the image data collected by the image acquisition device to obtain the corresponding training sample set, uses the constructed lightweight RT-DETR detection algorithm model to perform model training on the training sample set to obtain local model parameters, aggregates multiple clients based on the local model parameters to construct a visual detection model, and uses the visual detection model to realize machine vision detection, which breaks the barriers of data islands and realizes collaborative detection without data interaction, while greatly improving communication efficiency, reducing communication overhead, and alleviating computing pressure.
[0037] Example 2 Another aspect of the present invention is to provide a visual inspection model construction system. Figure 2 , which shows a visual detection model building system in a second embodiment of the present invention, the system includes: The data acquisition module 11 is used to acquire image data acquired by the image acquisition device and annotate the image data to form an image data set; a data processing module 12, configured to perform image data processing on the image dataset to increase sample diversity in the image dataset and obtain a corresponding training sample set; A model training module 13 is used to construct a lightweight RT-DETR detection algorithm model and input the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters; Furthermore, the model training module 13 includes: The network definition unit is used to define the backbone network of the model and use the lightweight GhostNet instead of the backbone network ResNet to reduce the initial complexity of the RT-DETR model; The model building unit is used to further lightweight the RT-DETR model by adopting a channel pruning method to obtain a lightweight RT-DETR detection algorithm model.
[0038] Furthermore, the model training module 13 also includes: A feature extraction unit is used to extract features from the training sample set using the backbone network of the lightweight RT-DETR detection algorithm model: ; in * is the convolution operation, b is the bias term, , n is the number of channels output feature map, is the convolution filter of the current layer. are the height and width of the output data respectively, k×k Convolution filters f The kernel size; Linear processing unit, used to Y Each feature in is subjected to a linear operation to obtain the required number of n feature maps, and s Ghost feature maps are generated according to the following function:
[0039] Where, y' i Indicates the first i feature maps, Φ i,j is a linear transformation function, expressed as a sequence i The feature map of j A linear transformation; The model training unit is used to use the GhostNet to perform data processing on the training sample set to obtain corresponding local model parameters.
[0040] The global aggregation module 14 is used to aggregate multiple clients based on the local model parameters, and use the global model weights obtained by aggregation to build a visual detection model, and use the visual detection model to implement machine vision detection.
[0041] Furthermore, the global aggregation module 14 includes: A distributed training unit, configured to perform multiple rounds of gradient descent on each client and introduce a federated averaging algorithm to perform distributed training on each of the clients; The global aggregation unit is used to perform global model aggregation on each of the clients and construct a visual detection model using the obtained global model weights.
[0042] The functions or operation steps implemented when the above modules and units are executed are substantially the same as those in the above method embodiments and will not be repeated here.
[0043] The visual detection model construction system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference can be made to the corresponding content in the aforementioned method embodiment.
[0044] Example 3 The present invention also provides a computer, see Figure 3 , shown is a computer in the third embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, the above-mentioned visual detection model construction method is implemented.
[0045] The memory 10 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer, such as the computer's hard disk. In other embodiments, the memory 10 may also be an external storage device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the memory 10 may include both an internal storage unit of the computer and an external storage device. The memory 10 can be used not only to store application software installed in the computer and various types of data, but also to temporarily store data that has been output or is about to be output.
[0046] Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs.
[0047] It should be pointed out that Figure 3 The structure shown does not constitute a limitation of the computer. In other embodiments, the computer may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0048] An embodiment of the present invention further provides a readable storage medium having a computer program stored thereon, which implements the above-mentioned visual detection model construction method when executed by a processor.
[0049] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.
[0050] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.
[0051] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.
[0052] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0053] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.
Claims
1. A method for constructing a visual inspection model, characterized in that: include: Acquiring image data captured by an image acquisition device, and annotating the image data to form an image data set; Performing image data processing on the image dataset to increase sample diversity in the image dataset to obtain a corresponding training sample set; Constructing a lightweight RT-DETR detection algorithm model, and inputting the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters; Multiple clients are aggregated based on the local model parameters, and a visual detection model is constructed using the global model weights obtained by aggregation, and machine vision detection is implemented using the visual detection model.
2. The method for constructing a visual detection model according to claim 1, wherein: The steps to build a lightweight RT-DETR detection algorithm model include: Define the backbone network of the model and use lightweight GhostNet instead of the backbone network ResNet to reduce the initial complexity of the RT-DETR model; Based on the RT-DETR model, the channel pruning method is used to further lightweight it to obtain a lightweight RT-DETR detection algorithm model.
3. The method for constructing a visual detection model according to claim 2, wherein: The step of inputting the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters includes: The backbone network of the lightweight RT-DETR detection algorithm model is used to extract features from the training sample set: ； in, * is the convolution operation, b is the bias term, , n is the number of channels output feature map, is the convolution filter of the current layer. are the height and width of the output data respectively, k×k Convolution filters f The kernel size; Through Y Each feature in is subjected to a linear operation to obtain the required number of n feature maps, and s Ghost feature maps are generated according to the following function: Where, y' i Indicates the first i feature maps, Φ i,j is a linear transformation function, expressed as a sequence i The feature map of j A linear transformation; The GhostNet is used to perform data processing on the training sample set to obtain corresponding local model parameters. The GhostNet is used to perform data processing on the training sample set to obtain corresponding local model parameters.
4. The method for constructing a visual detection model according to claim 1, wherein: The steps of aggregating multiple clients based on the local model parameters and constructing a visual detection model using the aggregated global model weights include: Perform multiple rounds of gradient descent on each client, and introduce the federated averaging algorithm to perform distributed training on each client; A global model aggregation is performed on each of the clients, and a visual detection model is constructed using the obtained global model weights.
5. The method for constructing a visual detection model according to claim 4, wherein: The calculation formula for the global model aggregation is: ； in, For the client The local model parameters of the local iteration multiple times, is the gradient of the local model, is the learning rate.
6. A visual inspection model construction system, characterized in that: include: A data acquisition module, configured to acquire image data acquired by an image acquisition device and annotate the image data to form an image data set; a data processing module, configured to perform image data processing on the image dataset to increase sample diversity in the image dataset and obtain a corresponding training sample set; A model training module is used to construct a lightweight RT-DETR detection algorithm model and input the training sample set into the lightweight RT-DETR detection algorithm model for model training to obtain local model parameters; A global aggregation module is used to aggregate multiple clients based on the local model parameters, and use the global model weights obtained by aggregation to build a visual detection model, and use the visual detection model to implement machine vision detection.
7. The visual inspection model construction system according to claim 6, characterized in that: The model training module includes: The network definition unit is used to define the backbone network of the model and use the lightweight GhostNet instead of the backbone network ResNet to reduce the initial complexity of the RT-DETR model; The model building unit is used to further lightweight the RT-DETR model by adopting a channel pruning method to obtain a lightweight RT-DETR detection algorithm model.
8. The visual inspection model construction system according to claim 7, characterized in that: The model training module also includes: A feature extraction unit is used to extract features from the training sample set using the backbone network of the lightweight RT-DETR detection algorithm model: ； in * is the convolution operation, b is the bias term, , n is the number of channels output feature map, is the convolution filter of the current layer. are the height and width of the output data respectively, k×k Convolution filters f The kernel size; Linear processing unit, used to Y Each feature in is subjected to a linear operation to obtain the required number of n feature maps, and s Ghost feature maps are generated according to the following function: Where, y' i Indicates the first i feature maps, Φ i,j is a linear transformation function, expressed as a sequence i The feature map of j A linear transformation; The model training unit is used to use the GhostNet to perform data processing on the training sample set to obtain corresponding local model parameters.
9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the visual detection model construction method according to any one of claims 1 to 5 is implemented.
10. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the visual detection model construction method according to any one of claims 1 to 5 is implemented.