Detection model operation method and device, computer equipment and readable storage medium

Abstract

The invention relates to a detection model operation method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an image sample based on an image acquired by a camera associated with lamp post equipment; under the condition that the number of the image samples reaches a reference number, training an initial model used for object detection into a reference detection model according to the image samples; the reference detection model supports the lamp post equipment to detect a target object; when the increment of the image sample reaches an increment threshold value, training the reference detection model as a target detection model according to the image sample; the target detection model is operated by the lamp pole equipment, and the target detection model supports the lamp pole equipment to detect the target object. By adopting the method, the training process and the application process of the model can be integrated, so that continuous model optimization is realized, and accuracy and robustness are guaranteed.

Description

Technical Field

[0001] This application relates to the field of smart light pole technology, and in particular to a detection model operation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of IoT technology, smart light pole technology has emerged. Smart light pole technology enables streetlights to not only be used for lighting, but also to serve as IoT terminals, realizing functions such as video acquisition and monitoring, so that a single light pole can have multiple functions.

[0003] In traditional techniques, the trained model is directly deployed on the light pole to run. However, due to the complex environment of the light pole, the detection model performs poorly. Summary of the Invention

[0004] Therefore, it is necessary to provide a detection model operation method, apparatus, computer equipment, computer-readable storage medium, and computer program product to address the above-mentioned technical problems, which can continuously optimize the model so that the target detection model can balance accuracy and robustness.

[0005] Firstly, this application provides a method for running a detection model, including:

[0006] Image samples are obtained based on images captured by cameras associated with the light pole equipment;

[0007] When the number of image samples reaches a reference number, the initial model for object detection is trained as a reference detection model based on the image samples; the reference detection model supports the light pole device in detecting target objects.

[0008] When the increment of the image samples reaches the increment threshold, the reference detection model is trained into a target detection model based on the image samples; the target detection model is run by the light pole device, and the target detection model supports the light pole device in detecting the target object.

[0009] In one embodiment, the step of training the reference detection model into a target detection model based on the image samples when the increment of the image samples reaches an increment threshold includes:

[0010] If the increment of the image sample reaches the first increment threshold, the reference detection model is trained into the first target detection model based on the image sample whose increment reaches the first increment threshold.

[0011] When the increment of the image sample reaches the i-th increment threshold, the (i-1)-th target detection model is trained into the i-th target detection model based on the image sample whose increment reaches the i-th increment threshold; the i-th target detection model is run by the light pole device; i is an integer greater than 1.

[0012] In one embodiment, the method further includes:

[0013] Based on the reference quantity and ratio parameters, the first incremental threshold is determined;

[0014] The i-th increment threshold is determined based on the number of image samples whose increment reaches the (i-1)-th increment threshold and the ratio parameter.

[0015] In one embodiment, the initial model includes initial model parameters for detecting target objects of a target size smaller than a size threshold;

[0016] The step of training the initial model for object detection into a reference detection model based on the image samples includes:

[0017] Based on the image samples, the initial model parameters are adjusted to obtain a reference detection model with reference detection parameters; wherein, the reference detection parameters are the result of adjusting the initial model parameters, and the reference detection parameters are used to detect the target object of the target size;

[0018] The step of training the reference detection model into a target detection model based on the image samples includes:

[0019] Based on the image samples, the reference detection parameters are adjusted to obtain a target detection model with target detection parameters; wherein, the target detection parameters are the result of adjusting the reference detection parameters, and the target detection parameters are used to detect target objects of the target size.

[0020] In one embodiment, obtaining image samples based on images captured by a camera associated with the light pole device includes:

[0021] A timed task for model training is sent to the light pole equipment; during the time period indicated by the timed task, the light pole equipment acquires images to be labeled according to the timed task.

[0022] The image to be labeled is obtained, and the target objects in the image to be labeled are detected and labeled to obtain the labeled image sample.

[0023] In one embodiment, the computing resources of the light pole device are positively correlated with the number of cameras associated with the light pole device; the number of cameras is negatively correlated with the time it takes for the number of image samples to reach the reference number and the time it takes for the increment of the image samples to reach the increment threshold.

[0024] Secondly, this application also provides a detection model running device, the device comprising:

[0025] The acquisition module is used to acquire image samples based on images captured by cameras associated with the light pole equipment;

[0026] A reference module is used to train an initial model for object detection into a reference detection model based on the image samples when the number of image samples reaches a reference number; the reference detection model supports the light pole device in detecting target objects.

[0027] The target module is used to train the reference detection model into a target detection model based on the image samples when the increment of the image samples reaches an increment threshold; the target detection model is run by the light pole device, and the target detection model supports the light pole device in detecting the target object.

[0028] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of running the detection model in any of the above embodiments.

[0029] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the detection model operation in any of the above embodiments.

[0030] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of running the detection model in any of the above embodiments.

[0031] The aforementioned detection model operation method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire image samples based on images captured by cameras associated with light pole devices. Since these image samples closely resemble the actual working environment of the light pole devices, using them for model training increases the model's accuracy and robustness. Furthermore, by first establishing a detection method specifically for light pole devices in a timely manner using a large number of image samples and reference models, a usable reference detection model for the light pole devices can be quickly established, facilitating accurate detection of target objects. Incremental thresholding is then used to balance the update frequency and duration of the detection model, and more image samples are employed to enhance robustness, allowing for flexible training of a more robust target detection model. Finally, the target detection model is run by the light pole device, an edge device, to more efficiently control the camera. Thus, by performing multi-stage model training using image samples from a specific environment, the model's accuracy and robustness are improved, while balancing the update frequency and duration, enabling the target detection model to be run by the light pole device, an edge device, for more efficient camera control. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a diagram illustrating the application environment of a detection model operation method in one embodiment;

[0034] Figure 2 This is a flowchart illustrating the detection model operation method in one embodiment;

[0035] Figure 3 This is a schematic diagram of the training process for an object detection model in one embodiment;

[0036] Figure 4 This is a structural block diagram of the detection model running device in one embodiment;

[0037] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] The detection model operation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the light pole device 102 communicates with the server 104 via a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104, or it can be located in the cloud or on another network server.

[0040] The light pole device 102 can be, but is not limited to, various IoT devices installed on streetlights, such as smart speakers, smart monitoring systems, smart lighting fixtures, and smart streetlights. The server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0041] In one exemplary embodiment, such as Figure 2 As shown, a method for running a detection model is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 206. Wherein:

[0042] Step 202: Obtain image samples based on images captured by the cameras associated with the light pole equipment.

[0043] Light pole devices are Internet of Things (IoT) devices used to control cameras mounted on light poles. These devices can be control boxes installed inside smart light poles or externally mounted on the pole. Compared to servers, light pole devices are considered edge devices, allowing for more efficient camera control.

[0044] A camera is an image acquisition device installed on a light pole, and it is controlled by the light pole equipment it is associated with. Each light pole equipment can control at least one camera, thus each light pole equipment corresponds to images captured by one or more cameras. The connection between the camera and the light pole equipment is electrical, meaning they directly transmit and control signals. Furthermore, the connection can also include a positional relationship, meaning the distance between the camera and the light pole equipment is within a certain range. The images captured by the camera can be multiple frames or video. When there are multiple cameras, the images captured are multiple video streams.

[0045] Image samples are images determined based on images captured by a camera. These image samples are used in the training processes of the initial model, reference detection model, and object detection model to obtain a detection model specifically for light pole devices. Image samples include at least labeled image samples with the target object already labeled, and may also include images without labeled target objects.

[0046] In some embodiments, the light pole device selects multiple frames from the video captured by the camera at preset time intervals and uploads them to the server; the server determines image samples based on the selected multiple frames. Therefore, the image acquisition process of the edge device uses fewer computing resources, and the server can determine image samples from more images based on the multiple frames transmitted by each light pole device.

[0047] In some embodiments, obtaining image samples based on images captured by cameras associated with the light pole device includes: capturing images from cameras associated with the light pole device and transmitting them to an annotation server; annotating target objects in the images to obtain annotated image samples; sending the annotated image samples to a quality inspection server; and using the quality-approved annotated image samples as samples for calculating the reference quantity and increment after the quality inspection server completes the quality inspection. Here, the annotation server is a physical or virtual server used for target object annotation for model training; the quality inspection server is a physical or virtual server used for quality inspection of the annotation results. Therefore, through the annotation and quality inspection of multiple image samples, the quality of the image samples can be made high.

[0048] Step 204: When the number of image samples reaches the reference number, the initial model used for object detection is trained into a reference detection model based on the image samples; the reference detection model supports the detection of target objects by the light pole device.

[0049] The reference count is a threshold number of reference detection model training samples. It represents the number of image samples that support the training of a reference detection model with a certain level of accuracy. The reference count is positively correlated with the target object category, which is at least one detectable object. For example, the target object category may include human detection or pedestrian detection, in which case the detection model is used to determine whether a human or pedestrian exists. The target object category may also include human boundary crossing detection, in which case the detection model is used to determine whether human boundary crossing behavior exists. Human boundary crossing behavior can be non-compliant behavior, such as behavior that affects light pole equipment, lighting fixtures, or other public facilities.

[0050] The initial model is a neural network model to be trained for object detection. The initial model can be a basic model framework or a pre-trained neural network model. For example, the initial model can be a detection model from the YOLO series; the initial model can be a pedestrian detection model pre-trained on a general dataset, and the pre-trained pedestrian detection model can be determined based on the YOLO series or other series of detection models; the initial model can be a detection model obtained by pruning a YOLO model, where pruning refers to cutting off the output layer or detection head to a certain size.

[0051] The reference detection model is a model specifically trained on the initial model based on a reference number of image samples. The reference detection model can be run by the light pole device to detect target objects directly based on images captured at the edge. The reference detection model has the same architecture as the initial model, but the specific values ​​of the parameters are adjusted. Therefore, in images captured by cameras associated with the light pole device, the reference detection model exhibits relatively high robustness and accuracy in detecting target objects.

[0052] In some embodiments, the situation where the number of image samples reaches the reference number means that, according to the detection cycle, it is determined whether the number of image samples is greater than the reference number; if so, the number of image samples has reached the reference number. The situation where the number of image samples reaches the reference number can also mean that the number of labeled or unlabeled image samples is greater than the reference number.

[0053] In some embodiments, training an initial model for object detection as a reference detection model based on image samples includes: dividing the image samples into a training set and a test set; adjusting the parameters of the initial model for object detection based on the training set to obtain an adjusted initial model; determining whether the adjusted initial model meets the stopping training condition for the reference detection model based on the test set; if so, determining the reference detection model based on the adjusted initial model; otherwise, using the adjusted initial model as the initial model for object detection and returning to the step of adjusting the parameters of the initial model for object detection based on the training set. Thus, iterative training using image samples ensures that the accuracy of the reference detection model is relatively high.

[0054] In some embodiments, training an initial model for object detection into a reference detection model based on image samples includes: dividing the image samples into multiple batches; and sequentially adjusting the initial model for object detection based on image samples of each frequency to obtain the reference detection model. Thus, through iterative training across multiple batches, the detection accuracy and robustness of the reference detection model are improved.

[0055] The reference detection model supports target object detection for light pole devices. This means that when the reference detection model is deployed on the light pole device, the light pole device can perform target object detection on the images captured by its own camera based on the deployed reference detection model, so as to identify the target objects in the images.

[0056] Optionally, the reference detection model can be deployed to the light pole equipment so that the light pole equipment can run the reference detection model. Deploying the reference detection model to the light pole equipment means that after the reference detection model has been trained, it is quantized onto the light pole equipment so that the light pole equipment can run the reference detection model. Quantizing the reference detection model onto the light pole equipment includes adjusting the parameters of the reference detection model according to the computing resources of the light pole equipment to obtain the quantized reference detection model; and distributing the quantized reference detection model to the light pole equipment.

[0057] The reference detection model is run by the light pole equipment. This means that the light pole equipment detects target objects in images captured by cameras managed by the light pole equipment itself, based on the reference detection model. Therefore, running the reference detection model through the edge-end light pole equipment results in relatively short data transmission lines and relatively high processing efficiency.

[0058] When a reference detection model is obtained based on a reference number of image samples, the training time of the initial model is relatively short, and a detection method specifically for light pole equipment can be formed more quickly, thereby ensuring that the light pole equipment can detect target objects more quickly.

[0059] Step 206: When the increment of image samples reaches the increment threshold, the reference detection model is trained into a target detection model based on the image samples; the target detection model is run by the light pole device and supports the light pole device in detecting target objects.

[0060] The increment of image samples indicates the growth of image samples. It represents the increase in the number of image samples relative to the number of image samples during the previous model training. The increment can be a specific number or a percentage increase based on the number of image samples, such as 20%, 30%, or other percentages.

[0061] The increment threshold is a threshold used to obtain the number of data points for the object detection model. When the increment reaches the increment threshold, the increase in image samples supports the robustness optimization of the detection model. Robustness refers to the ability of the object detection model to accurately detect the target object when the image captured by the camera associated with the light pole device is disturbed. The higher the robustness, the better the object detection model can detect the target object when the image captured by the camera associated with the light pole device is more disturbed; the lower the robustness, the lower the accuracy of the object detection model in detecting the target object when the image captured by the camera associated with the light pole device is disturbed.

[0062] The increment threshold is positively correlated with the target object category; the more target object categories there are, the larger the increment threshold; the fewer target object categories there are, the smaller the increment threshold. The increment threshold and increment are matched: when the increment is a specific value, the increment threshold is a specific value; when the increment is a proportion, the increment threshold is a proportion. Using an increment threshold ensures that the number of image samples during retraining is within an appropriate range, balancing the update frequency and update duration of the detection model.

[0063] The object detection model is trained on a reference detection model using incrementally updated image samples. This model can be run by light pole devices to detect objects based on images captured at the edge. Furthermore, the object detection model exhibits higher robustness in object detection. While the architecture of the object detection model is the same as the reference detection model, the specific parameter values ​​are adjusted, and a larger number of image samples are used. Therefore, the object detection model demonstrates relatively higher robustness in object detection within images captured by cameras associated with the light pole devices.

[0064] In some embodiments, the situation where the increase in image samples reaches the increment threshold means that, according to the detection cycle, it is determined whether the number of images added to the image samples relative to the previous training round is greater than the increment threshold for the current training. If so, the increase in image samples reaches the increment threshold. The situation where the increase in image samples reaches the increment threshold can also mean that the proportion of increasing labeled or unlabeled image samples is greater than the increment threshold.

[0065] In some embodiments, training a reference detection model into an object detection model based on image samples includes: dividing the incremental image samples into a training set and a test set; adjusting the parameters of the reference detection model based on the training set to obtain an adjusted reference detection model; determining whether the adjusted reference detection model meets the stopping training condition for the object detection model based on the test set; if so, determining the object detection model based on the adjusted reference detection model; if not, using the adjusted reference detection model as the reference detection model for object detection, and then returning to the step of adjusting the parameters of the reference detection model for object detection based on the training set. Thus, iterative training using image samples results in a relatively high robustness of the object detection model.

[0066] In some embodiments, training a reference detection model into an object detection model based on image samples includes: dividing the image samples into multiple batches; and sequentially adjusting the reference detection model according to image samples of each frequency to obtain the object detection model. Thus, through iterative training across multiple batches, the robustness of the object detection model is improved.

[0067] The object detection model is run by the light pole equipment, which requires that the object detection model be deployed to the light pole equipment. Deploying the object detection model to the light pole equipment means that after the object detection model has been trained, it is quantized and distributed to the light pole equipment so that the light pole equipment can run the object detection model. Quantizing the object detection model to the light pole equipment includes adjusting the parameters of the object detection model according to the computing resources of the light pole equipment to obtain the quantized object detection model; and distributing the quantized object detection model to the light pole equipment.

[0068] The target detection model is run by the light pole equipment, meaning that the light pole equipment detects target objects in images captured by cameras managed by the light pole equipment itself, based on the target detection model. Therefore, running the target detection model through the edge-end light pole equipment results in relatively short data transmission lines and relatively high processing efficiency.

[0069] The object detection model supports target object detection on light pole devices. This means that when the object detection model is deployed on a light pole device, the device can use this model to detect target objects in images captured by its own camera, thus identifying the target objects within the images. Because the object detection model is trained incrementally based on image samples, its robustness is increased, allowing the light pole device to detect target objects more accurately.

[0070] In the case of training the reference detection model based on incremental image samples, the image samples are determined based on the cameras associated with the light pole equipment. These image samples are relatively close to the actual working environment of the light pole equipment, which helps to enhance robustness. Since the cameras associated with the light pole equipment are set on the light pole, and the actual target object is located around the light pole, the environment around the light pole is an uncontrollable factor. Therefore, the interference of the images captured by the camera has certain particularities, thus requiring more image samples to enhance robustness.

[0071] In one example, after the reference detection model has been trained, image samples are continuously acquired to increase the number of image samples, forming an increment. The reference detection model is then further optimized using the original number of image samples and the increment. Similarly, after the object detection model has been trained but the training stopping condition has not been met, image samples are continuously acquired to further increase the number of image samples, forming the i-th increment. The object detection model is then further optimized using the original number of image samples and the i-th increment. The training stopping condition can be either a set time interval or a condition that robustness is met.

[0072] In the above detection model operation method, image samples are obtained based on images captured by cameras associated with the light pole equipment. This ensures that the image samples closely resemble the actual working environment of the light pole equipment, thus increasing the accuracy and robustness of the model during training. When the number of image samples reaches a reference number, the initial model for object detection is trained as a reference detection model based on these image samples. Since the training time for the reference detection model is relatively short and it can quickly develop a detection method specifically for light pole equipment, it ensures that a usable reference detection model exists for the light pole equipment, facilitating accurate detection of target objects. Furthermore, when the increment of image samples reaches an increment threshold, the reference detection model is trained as a target detection model based on the image samples, and the target detection model is deployed to the light pole equipment. This approach combines the use of the increment threshold with consideration for the update frequency and duration of the detection model, while utilizing more image samples to enhance robustness and flexibly train a more robust target detection model. Finally, the target detection model is deployed to the light pole equipment; by running the target detection model on this edge device, the light pole equipment can more efficiently control the camera.

[0073] In one exemplary embodiment, such as Figure 3 As shown, when the increment of image samples reaches the increment threshold, the reference detection model is trained into a target detection model based on the image samples, including steps 302 to 304. Wherein:

[0074] Step 302: When the increment of the image sample reaches the first increment threshold, the reference detection model is trained into the first target detection model based on the image sample whose increment reaches the first increment threshold.

[0075] The first increment threshold is the increment threshold used during the initial training of the reference detection model. When the increment reaches the first increment threshold, the number of image samples includes the sum of the reference number and the number corresponding to the first increment threshold; at this point, the number of image samples has substantially increased, thus the robustness of the first object detection model is higher than that of the reference detection model. The first object detection model is an object detection model trained based on the reference detection model. For example, with a reference number of 5000 and an increment of 1000 images, the reference detection model is trained into the first object detection model based on 6000 image samples.

[0076] In some embodiments, training a reference detection model into a first object detection model based on image samples whose increment reaches a first increment threshold includes: dividing the image samples whose increment reaches the first increment threshold into a training set and a test set; adjusting the parameters of the reference detection model based on the training set to obtain an adjusted reference detection model; determining whether the adjusted reference detection model meets the stopping training condition of the first object detection model based on the test set; if so, determining the first object detection model based on the adjusted reference detection model; if not, using the adjusted reference detection model as the reference detection model for object detection, and returning to the step of adjusting the parameters of the reference detection model for object detection based on the training set. Thus, iterative training using image samples whose increment reaches the first increment threshold results in relatively high robustness of the first object detection model.

[0077] In some embodiments, training a reference detection model into a first object detection model based on image samples whose increments reach a first increment threshold includes: dividing the image samples whose increments reach the first increment threshold into multiple batches; and sequentially adjusting the reference detection model based on image samples whose increments reach the first increment threshold at each frequency to obtain the first object detection model. Thus, through iterative training with multiple batches, the detection robustness of the first object detection model is improved.

[0078] In some embodiments, training a reference detection model into a first target detection model based on image samples whose increment reaches a first increment threshold includes: adjusting the reference detection parameters based on the image samples to obtain a first target detection model with first target detection parameters; wherein the first target detection parameters are the result of adjusting the reference detection parameters, and the first target detection parameters are used to detect target objects of target size.

[0079] Step 304: When the increment of the image sample reaches the i-th increment threshold, the (i-1)-th target detection model is trained into the i-th target detection model based on the image sample whose increment reaches the i-th increment threshold; the i-th target detection model is run by the light pole device; i is an integer greater than 1.

[0080] 'i' represents the current stage of the object detection model's optimization. The first object detection model is the initial model, trained based on a reference model. From the second model onwards, each object detection model is trained based on the model from the previous stage. The current stage is the stage used by the object detection model in the current training round.

[0081] The i-th increment threshold is the increment threshold used during the training of the (i-1)-th object detection model. When the increment reaches the i-th increment threshold, the number of image samples includes the sum of the reference number and the number corresponding to the i-th increment threshold; at this point, the number of image samples has substantially increased, thus the robustness of the i-th object detection model is higher than that of the reference detection model. The i-th object detection model is the object detection model trained based on the (i-1)-th object detection model. The first (i-1)-th object detection model is the first object detection model.

[0082] The first increment threshold and the i-th increment threshold can be specific quantities, and they can be the same or different. The first increment threshold and the i-th increment threshold can also be ratio values, and the cases where the increment of the image sample reaches the first increment threshold and the cases where the increment of the image sample reaches the i-th increment threshold can be determined in the same or different ways.

[0083] The i-th target detection model is run by the light pole equipment. At this time, the target detection model run by the light pole equipment is the latest optimized model, which can support the highly robust detection of target objects.

[0084] In some embodiments, training the first object detection model into the i-th object detection model based on image samples whose increment reaches the i-th increment threshold includes: dividing the image samples whose increment reaches the i-th increment threshold into a training set and a test set; adjusting the parameters of the first object detection model based on the training set to obtain an adjusted first object detection model; determining whether the adjusted first object detection model meets the stopping training condition of the i-th object detection model based on the test set; if so, determining the i-th object detection model based on the adjusted first object detection model; if not, using the adjusted first object detection model as the first object detection model for object detection, and then returning to the step of adjusting the parameters of the first object detection model for object detection based on the training set. Thus, iterative training using image samples whose increment reaches the i-th increment threshold results in relatively high robustness of the i-th object detection model.

[0085] In some embodiments, training the first object detection model into the i-th object detection model based on image samples whose increment reaches the i-th increment threshold includes: dividing the image samples whose increment reaches the i-th increment threshold into multiple batches; and sequentially adjusting the first object detection model according to the image samples whose increment reaches the i-th increment threshold at each frequency to obtain the i-th object detection model. Thus, through iterative training with multiple batches, the detection robustness of the i-th object detection model is improved.

[0086] In some embodiments, training a reference detection model into an i-th target detection model based on image samples whose increment reaches the i-th increment threshold includes: adjusting the reference detection parameters based on the image samples to obtain an i-th target detection model with i-th target detection parameters; wherein the i-th target detection parameters are the result of adjusting the reference detection parameters, and the i-th target detection parameters are used to detect target objects of target size.

[0087] In this embodiment, the object detection model is trained in segments based on each incremental threshold. Therefore, each training segment will not cause the model to train too frequently due to too small an increment, thus avoiding excessive consumption of server resources, nor will it cause the model update time to be too long due to too large an increment. As a result, the update frequency and update time of the object detection model can be controlled more precisely, so that a more robust model can be used at each stage.

[0088] In an exemplary embodiment, the method further includes: determining a first incremental threshold based on a reference quantity and a scaling parameter; and determining an i-th incremental threshold based on the number of image samples whose increment reaches the (i-1)-th incremental threshold and the scaling parameter.

[0089] The scaling parameter is a proportional value used to determine the increment threshold. It indicates the extent to which the number of samples that increase robustness increases with each increment reaching the increment threshold. Since increasing robustness becomes increasingly difficult, the model is trained on image samples from the previous stage to control for any increase in robustness.

[0090] In some embodiments, the first incremental threshold can be determined based on the product or ratio of the reference quantity and the scale parameter. For example, if the reference quantity is 5000 and the scale parameter is 20%, then the first incremental threshold is 1000 images.

[0091] The number of image samples whose increment reaches the (i-1)th increment threshold refers to the cumulative number of images acquired at the moment the increment reaches the (i-1)th increment threshold. The cumulative number of images acquired includes the total number of image samples acquired by the server from the initial moment until the moment the increment reaches the (i-1)th increment threshold. Taking the moment when the increment reaches the second increment threshold as an example, with a reference number of 4000 and a ratio parameter of 25%, the number of image samples whose increment reaches the second increment threshold is 6250.

[0092] In some embodiments, the i-th increment threshold is determined based on the number of image samples whose increment reaches the (i-1)-th increment threshold and the product or ratio of the scaling parameter. For example, if the number of samples at the current time is 6000 and the scaling parameter is 20%, then the i-th increment threshold is 1000 images.

[0093] In this embodiment, based on the increment of image samples during the previous stage of model training, the threshold for each increment is dynamically changed to ensure that the robustness of the model increases with each training iteration through a proportional parameter. This allows for more accurate control over the degree of increment change, ensuring that the model's robustness increases gradually.

[0094] In some embodiments, the initial model includes initial model parameters for detecting target objects of a target size smaller than a size threshold. Training the initial model for object detection into a reference detection model based on image samples includes: adjusting the initial model parameters based on the image samples to obtain a reference detection model with reference detection parameters; wherein the reference detection parameters are the result of adjusting the initial model parameters, and the reference detection parameters are used to detect target objects of a target size.

[0095] Correspondingly, based on the image samples, the reference detection model is trained into an object detection model, including: adjusting the reference detection parameters based on the image samples to obtain an object detection model with object detection parameters; wherein, the object detection parameters are the result of adjusting the reference detection parameters, and the object detection parameters are used to detect the target object of the target size.

[0096] The size threshold represents the size threshold for initial model pruning. The initial model can detect both target objects smaller than the size threshold and target objects larger than the size threshold. However, after setting the target size through the size threshold, target objects larger than the size threshold are not detected. The corresponding detection head and output layer are omitted, and the focus is instead on the detection process of small-sized target objects.

[0097] When a camera is mounted on a lamppost, the target object detected by the camera is relatively small in size within the frame. In this case, the size that the detection model can detect is limited to the target size, so as to detect target objects with smaller receptive fields, and omit the detection of target objects with larger sizes and larger receptive fields.

[0098] In some embodiments, adjusting the initial model parameters based on image samples to obtain a reference detection model with reference detection parameters includes: adjusting the initial model parameters that do not contain large target detection head parameters and medium target detection head parameters based on image samples to obtain a reference detection model with reference detection parameters. Thus, the initial model parameters do not contain output layer-related parameters with sizes larger than a size threshold, thereby omitting the corresponding size detection process during training and achieving model pruning.

[0099] In some embodiments, adjusting the initial model parameters based on image samples to obtain a reference detection model with reference detection parameters includes: adjusting the YOLO model parameters that do not contain large target detection head parameters and medium target detection head parameters based on image samples to obtain a reference detection model with reference detection parameters.

[0100] In some embodiments, adjusting reference detection parameters based on image samples to obtain a target detection model with target detection parameters includes: adjusting reference detection parameters that do not contain large or medium target detection head parameters based on image samples to obtain a target detection model with target detection parameters. Thus, the reference model parameters do not contain output layer-related parameters with sizes larger than a size threshold, thereby omitting the corresponding size detection process during training and achieving efficient training after model pruning.

[0101] In this embodiment, since there is a large height difference between the camera of the light pole device and the plane where the target object is located, the size of the target object in the image detected by the camera is usually relatively small. Therefore, by pruning the model, only the target object detection network smaller than the size threshold is retained to improve training efficiency without affecting the detection accuracy.

[0102] In some embodiments, obtaining image samples based on images captured by cameras associated with the light pole device includes: sending a timed task for model training to the light pole device; during the time period indicated by the timed task, the light pole device captures images to be labeled according to the timed task; obtaining the images to be labeled, detecting and labeling target objects in the images to be labeled, and obtaining labeled image samples.

[0103] The timed task for model training is performed by the light pole equipment, indicating its image acquisition during the training process. Due to the relatively large number of light poles and their primary outdoor application, they are subject to more random events, necessitating longer image acquisition periods to ensure robustness. Therefore, by sending timed tasks, the timeframe for image sample acquisition by the light pole equipment is controlled over a longer period, ensuring continuous image sample acquisition. This controls the training duration of the reference detection model and the object detection model, as well as the timing of training for each model. This allows for an increase in the total training time for at least two detection models.

[0104] For example, a scheduled task refers to generating a human boundary crossing model during the period from January 10, 2025 to May 10, 2025. The human boundary crossing model is a neural network model used to determine whether a pedestrian has crossed the boundary.

[0105] The images to be labeled are at least a portion of the images captured by the light pole equipment. These images may or may not contain the target object. Optionally, if a reference detection model is deployed on the light pole equipment, the images captured by the camera can be detected first using the reference detection model deployed on the light pole equipment, and the images containing the target object can be used as the images to be labeled. This avoids an excessive number of images to be labeled, thus ensuring moderate transmission efficiency. Alternatively, the images captured by the camera can be detected first using the target detection model deployed on the light pole equipment, and the images containing the target object can be used as the images to be labeled. This approach provides better robustness for image labeling, avoiding an excessive number of images to be labeled, thus ensuring moderate transmission efficiency, and preventing omissions.

[0106] Annotated image samples are images labeled with the target object. Annotated images can carry annotation information or label information of the target object, as well as the location information of the target object, so as to train a target object detection model.

[0107] In some embodiments, acquiring an image to be labeled, detecting and labeling target objects in the image to be labeled, and obtaining labeled image samples includes: after the light pole device sends the image to be labeled to a pre-labeling platform, the pre-labeling platform detects target objects in the image to be labeled to obtain a pre-labeled image; after the pre-labeling platform sends the pre-labeled image to a quality inspection platform, and the quality inspection platform determines that the target objects in the pre-labeled image meet the labeling requirements, the pre-labeled image that meets the labeling requirements is used as a labeled image sample. The pre-labeling platform is a program that detects target objects based on a general model; the pre-labeled image is an image of the target objects labeled by the pre-labeling platform; the quality inspection platform is a program that detects labeling information using a quality inspection model; the pre-labeling platform and the quality inspection platform can run on a terminal or a server.

[0108] In this embodiment, the time period for image sample collection by the light pole equipment is controlled over a long period by sending timed tasks, ensuring that the image sample collection process is continuous. This controls the training time of various detection models and the timing of model training. Subsequently, the server performs target object detection and annotation based on the image samples to be labeled, thereby accurately obtaining image samples through greater computing power and richer light pole equipment data.

[0109] In some embodiments, the computing resources of the light pole device are positively correlated with the number of cameras associated with the light pole device; the number of cameras is negatively correlated with the time it takes for the number of image samples to reach a reference number and the time it takes for the increment of image samples to reach an increment threshold.

[0110] The number of cameras refers to the number of cameras installed on the light pole. Each camera can independently capture its corresponding image, thus forming various video feeds. Computing resources refer to the computing power of the light pole device. Computing resources can be attributes of the MCU, memory, cache, or CPU.

[0111] The number of cameras is positively correlated with the computing resources of the light pole equipment. This means that the more computing resources the light pole equipment has, the more cameras it can control; conversely, the fewer computing resources the light pole equipment has, the fewer cameras it can control.

[0112] The number of cameras is negatively correlated with the time it takes for the number of image samples to reach the reference number and the time it takes for the increase in the number of image samples to reach the increment threshold. In other words, the more cameras there are, the more images each camera on each light pole device can collect, the higher the efficiency of image sample acquisition, and the faster the number of image samples can reach the parameter number and the increment threshold. Conversely, the fewer cameras there are, the fewer images each camera on each light pole device can collect, the lower the efficiency of image sample acquisition, and the slower the number of image samples can reach the parameter number and the increment threshold.

[0113] In this embodiment, the detection model is trained in stages, and the training and application processes can be implemented simultaneously. Since a greater number of cameras requires more computing power, the image acquisition speed increases, thus balancing the cost of computing resources and image acquisition speed. Therefore, with higher computing resources, faster image acquisition improves model training efficiency, and the training cost is primarily attributed to computing resources. Conversely, with lower computing resources and slower image acquisition speeds, model training efficiency is relatively lower, and the training cost mainly lies in the cost of long-term training. Thus, the cost of computing power and efficiency can be adaptively balanced according to requirements.

[0114] In one example, target objects in the initial image samples are labeled to obtain object labeling information corresponding to the initial image samples. A model is trained based on the initial image samples and their corresponding object labeling information to obtain a reference detection model for the target objects. The reference detection model is distributed to the light pole devices for operation, and incremental image samples are acquired based on the light pole devices. Incremental image samples include a reference number of initial image samples, and additional image samples added after reaching the reference number. The reference detection model is trained based on the incremental image samples and their corresponding object labeling information to obtain the initial target detection model for the target objects. The initial target detection model is quantized according to the computing resources of the light pole devices, and the quantized initial target detection model is distributed to the light pole devices for operation. While the initial target detection model is running, additional incremental image samples can be acquired based on the light pole devices. These additional incremental image samples include the incremental image samples themselves, as well as image samples where the increment reaches each increment threshold. The reference detection model is trained based on the incremental image samples and their corresponding object labeling information to obtain the initial target detection model for the target objects.

[0115] In another example, a continuous model optimization system is designed for smart light pole applications. This system fully automates sample collection, pre-labeling, pre-labeled sample quality checks, model training, and model quantization. This embodiment focuses on the fully automated model training, quantization, and deployment. The system automatically performs model training, quantization, and deployment according to the issued model training timing rules. Therefore, a continuous model optimization system is designed for smart light pole applications. This system can generate models supporting edge device systems with different AI chips while reducing human resource investment. Furthermore, the model can be continuously optimized.

[0116] This implementation case system is divided into six parts in general, namely: issuing timed tasks for model training, collecting pre-labeled samples from edge devices, verifying pre-labeled samples, model training, model quantization, and model deployment.

[0117] I. Issue the timed task for model training

[0118] The main function of this module is to install a server on the pole platform and issue a scheduled task for model training to multiple light pole devices. For example, generating a human body boundary crossing model from January 10, 2025 to May 10, 2025. During the time period specified in the scheduled task, the light pole devices collect images to be labeled according to the scheduled task.

[0119] II. Pre-labeled samples collected by edge devices

[0120] After receiving the task from Module 1, the light pole device responds as follows: 1) Each light pole device acquires the real-time video stream from its associated camera and captures images from the video at time intervals. 2) The captured images are uploaded to the AI ​​self-evolution platform's server so that the AI ​​self-evolution platform can pre-annotate and verify the images.

[0121] III. Verification of Pre-labeled Samples

[0122] 1) Using an initial model for general object detection, target detection is performed on the human body. The detected information is written into an XML file containing labeled training samples. This information includes category information and coordinate information. Category information is used to determine the pedestrian and the background, while coordinate information is used to determine the pedestrian's location, which can be the location of the region of interest. 2) The pre-labeled information obtained in step 1) of this module undergoes quality inspection using the pre-labeled sample quality control system on a cloud-based evolution platform. This ensures the quality of the labeled samples.

[0123] IV. Model Training

[0124] The main function of this module is to train a model using collected training samples to obtain an object detection model. Considering the actual application scenario of the company, namely that the camera is installed on a lamppost and the detected targets are relatively small objects in the image, the YOLOv4 model is selected for this part, and the output layer is pruned for medium and / or large object detection. For example, only the 76×76 detection head is retained for detection.

[0125] Specifically, 1) During the first model training, execute the specific implementation of step 204. At this time, it is necessary to determine whether the number of collected samples has reached a certain amount: such as 5000 images. When the set amount is reached, the background automatically starts the model training task to train the model. If the number of image samples has not reached the set amount, the background continues to wait.

[0126] 2) After the first model training is completed, execute the specific implementation of step 206. At this time, it is necessary to determine whether the incremental amount of collected samples has reached a certain amount, such as 20% of the previous training samples. That is: if the previous training samples were 5000, the current training samples must reach 5000*(1+0.2)=6000 before the current model training can start.

[0127] V. Model Quantization

[0128] The main content of this module is to quantize the model obtained in Module 4 based on the content of different edge chips. A cloud-based evolution platform server is deployed with quantization environments supporting N different chip models. Each quantization environment is deployed in its own container (Docker) environment, and the cloud-based evolution platform implements model quantization via scripts. The operators and hardware acceleration vary, and the number of image paths differs. Higher computing power allows for more image paths, increasing the number of training samples; up to 5000 images are supported; higher computing power results in higher iteration efficiency and shorter training time; lower computing power results in lower hardware costs but longer training time.

[0129] VI. Model Deployment

[0130] After Module 5 is completed, N models will be generated, each adapted to a specific chip. The cloud-based self-evolution platform will then distribute the models based on the type of AI chip on each light pole device.

[0131] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0132] Based on the same inventive concept, this application also provides a detection model running apparatus for implementing the detection model running method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more detection model running apparatus embodiments provided below can be found in the limitations of the detection model running method described above, and will not be repeated here.

[0133] In one exemplary embodiment, such as Figure 4 As shown, a detection model running device is provided, the device comprising:

[0134] The acquisition module 402 is used to acquire image samples based on images captured by cameras associated with the light pole device;

[0135] The reference module 404 is used to train an initial model for object detection into a reference detection model based on the image samples when the number of image samples reaches a reference number; the reference detection model supports the light pole device in detecting the target object;

[0136] The target module 406 is used to train the reference detection model into a target detection model based on the image samples when the increment of the image samples reaches the increment threshold; the target detection model is run by the light pole device and the target detection model supports the light pole device in detecting the target object.

[0137] In one embodiment, the target module 406 is configured to:

[0138] If the increment of the image sample reaches the first increment threshold, the reference detection model is trained into the first target detection model based on the image sample whose increment reaches the first increment threshold.

[0139] When the increment of the image sample reaches the i-th increment threshold, the (i-1)-th target detection model is trained into the i-th target detection model based on the image sample whose increment reaches the i-th increment threshold; the i-th target detection model is run by the light pole device; i is an integer greater than 1.

[0140] In one embodiment, the target module 406 is configured to:

[0141] Based on the reference quantity and ratio parameters, the first incremental threshold is determined;

[0142] The i-th increment threshold is determined based on the number of image samples whose increment reaches the (i-1)-th increment threshold and the ratio parameter.

[0143] In one embodiment, the initial model includes initial model parameters for detecting target objects of a target size smaller than a size threshold;

[0144] The reference module 404 is used for:

[0145] Based on the image samples, the initial model parameters are adjusted to obtain a reference detection model with reference detection parameters; wherein, the reference detection parameters are the result of adjusting the initial model parameters, and the reference detection parameters are used to detect the target object of the target size;

[0146] The target module 406 is used for:

[0147] Based on the image samples, the reference detection parameters are adjusted to obtain a target detection model with target detection parameters; wherein, the target detection parameters are the result of adjusting the reference detection parameters, and the target detection parameters are used to detect target objects of the target size.

[0148] In one embodiment, the acquisition module 402 is configured to:

[0149] A timed task for model training is sent to the light pole equipment; during the time period indicated by the timed task, the light pole equipment acquires images to be labeled according to the timed task.

[0150] The image to be labeled is obtained, and the target objects in the image to be labeled are detected and labeled to obtain the labeled image sample.

[0151] In one embodiment, the computing resources of the light pole device are positively correlated with the number of cameras associated with the light pole device; the number of cameras is negatively correlated with the time it takes for the number of image samples to reach the reference number and the time it takes for the increment of the image samples to reach the increment threshold.

[0152] Each module in the aforementioned detection model operating device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0153] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external lighting equipment via a network connection. When the computer program is executed by the processor, it implements a detection model operation method.

[0154] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0157] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of detecting model runs, the method comprising: The method comprises: acquiring image samples based on images collected by a camera associated with a lamp pole device; in a case where a quantity of the image samples reaches a reference quantity, training an initial model for object detection into a reference detection model according to the image samples; the reference detection model supports the lamp pole device to detect a target object; in a case where an increment of the image samples reaches an increment threshold, training the reference detection model into a target detection model according to the image samples; the target detection model is run by the lamp pole device, and the target detection model supports the lamp pole device to detect the target object.

2. The method of claim 1, wherein, The method further comprises: in a case where the increment of the image samples reaches a first increment threshold, training the reference detection model into a first target detection model based on image samples whose increment reaches the first increment threshold; in a case where the increment of the image samples reaches an i th increment threshold, training an (i-1) th target detection model into an i th target detection model based on image samples whose increment reaches the i th increment threshold; the i th target detection model is run by the lamp pole device; i is an integer greater than 1.

3. The method of claim 2, wherein, The method further comprises: determining the first increment threshold based on the reference quantity and a proportion parameter; determining the i th increment threshold based on a quantity of image samples whose increment reaches the (i-1) th increment threshold and the proportion parameter.

4. The method of claim 1, wherein, The initial model comprises initial model parameters for detecting a target object of a target size, the target size being less than a size threshold; The method further comprises: adjusting the initial model parameters according to the image samples to obtain a reference detection model with reference detection parameters; the reference detection parameters are results of adjustment of the initial model parameters, and the reference detection parameters are used to detect the target object of the target size; The method further comprises: adjusting the reference detection parameters according to the image samples to obtain a target detection model with target detection parameters; the target detection parameters are results of adjustment of the reference detection parameters, and the target detection parameters are used to detect the target object of the target size.

5. The method of claim 1, wherein, The method further comprises: sending a model training timing task to the lamp pole device; in a time period indicated by the timing task, the lamp pole device collects to-be-labeled images according to the timing task; acquiring the to-be-labeled images, detecting and labeling target objects in the to-be-labeled images to obtain labeled image samples.

6. The method of claim 1, wherein, The computing resource of the lamp pole device is positively related to the number of cameras associated with the lamp pole device; the number of cameras is negatively related to a time length for the image samples to reach the reference number, and a time length for an increment of the image samples to reach an increment threshold.

7. A detection model running apparatus characterized by comprising: The apparatus comprises: An acquisition module configured to acquire image samples based on images captured by cameras associated with a lamp pole device; A reference module configured to, in a case where a number of the image samples reaches a reference number, train an initial model for object detection into a reference detection model according to the image samples; the reference detection model supports the lamp pole device to detect a target object; A target module configured to, in a case where an increment of the image samples reaches an increment threshold, train the reference detection model into a target detection model according to the image samples; the target detection model is run by the lamp pole device, and the target detection model supports the lamp pole device to detect the target object.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 6.

Images