A template map automatic updating method, device, equipment and storage medium

Abstract

The present application relates to a kind of template graph automatic updating method, device, equipment and storage medium, it relates to image recognition field, the method comprises: frame image is intercepted from the video object of pre-set as to be examined graph, and the template graph corresponding to the to-be-examined graph is obtained;Based on the template graph corresponding to the to-be-examined graph, whether the to-be-examined graph exists foreign matter is judged by pre-set comparison model;If the to-be-examined graph does not exist foreign matter, according to the template graph corresponding to to-be-examined graph and to-be-examined graph, update template graph is calculated.Solve the way of the existing fixed template graph to detect foreign matter, is susceptible to environmental change and produces the problem of false detection result, realizes that template graph is dynamically updated with environmental change, on the one hand, the false detection result of updated template graph reduces the occurrence, on the other hand, reduce false detection result greatly improve the accuracy of template graph using comparison model to update thereafter, reduce the result of false detection and the accuracy of template graph update two mutually positive feedback.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and in particular to a method, apparatus, device, and storage medium for automatically updating template images. Background Technology

[0002] In recent years, with the rapid development of deep learning, computer vision has become the most important technology in the field of artificial intelligence. Various image recognition models (networks) have emerged in large numbers.

[0003] By using differential algorithms based on traditional image processing algorithms or Siamese network models based on deep learning, it is possible to identify whether there are foreign objects in the image under test relative to the template image.

[0004] However, both differential algorithms and Siamese network models place high demands on the selection of template images. Incorrect template image selection directly impacts the identification results of foreign objects in the image to be inspected. In existing technologies, template images are typically selected manually and are usually fixed after selection. Since the shooting environment is constantly changing, using a fixed template image often leads to false detections due to environmental variations. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for automatic updating of template images, which solves the problem that existing methods of detecting foreign objects using fixed template images are prone to false detections due to environmental changes.

[0006] In a first aspect, the present invention provides a method for automatically updating a template diagram, the method comprising:

[0007] Frame images are extracted from a preset video object as the image to be inspected, and the template image corresponding to the image to be inspected is obtained;

[0008] Based on the template image corresponding to the image to be inspected, a preset comparison model is used to determine whether there are foreign objects in the image to be inspected.

[0009] If there are no foreign objects in the image to be inspected, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected.

[0010] In a second aspect, the present invention provides an automatic template drawing update apparatus, including a unit for performing the automatic template drawing update method as described in any embodiment of the first aspect.

[0011] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0012] Memory, used to store computer programs;

[0013] When a processor executes a program stored in memory, it implements the steps of the automatic template drawing update method described in any embodiment of the first aspect.

[0014] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the template diagram automatic update method as described in any embodiment of the first aspect.

[0015] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art:

[0016] The method provided in this embodiment of the invention, based on a template image corresponding to the image to be inspected, determines whether there are foreign objects in the image to be inspected using a preset comparison model. When no foreign objects are found in the image to be inspected, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected. Updating the template image using an image to be inspected without foreign objects allows the template image to be continuously corrected along with the image to be inspected, making the updated template image more consistent with the current shooting environment and effectively avoiding false detections due to environmental changes. On the one hand, the updated template image reduces the occurrence of false detections; on the other hand, reducing false detections greatly improves the accuracy of subsequent template image updates using the comparison model. The reduction in false detections and the accuracy of template image updates mutually reinforce each other. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an automatic template diagram updating method provided in Embodiment 1 of the present invention;

[0020] Figure 2 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention;

[0022] Figure 4 This is a flowchart illustrating an automatic template diagram updating method provided in Embodiment 2 of the present invention;

[0023] Figure 5 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of the structure of an automatic template image updating device provided in Embodiment 1 of the present invention;

[0025] Figure 7 This is a schematic diagram of the structure of an automatic template image updating device provided in Embodiment 2 of the present invention;

[0026] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Example 1

[0029] Figure 1 This is a flowchart illustrating an automatic template diagram updating method provided by an embodiment of the present invention. Specifically, this embodiment of the present invention proposes an automatic template diagram updating method; see [link to documentation]. Figure 1 The automatic update method for the template diagram includes the following steps S101-S103.

[0030] S101, extract a frame image from a preset video object as the image to be inspected, and obtain the template image corresponding to the image to be inspected.

[0031] In practice, the video object refers to the video stream or video file acquired by the acquisition device. A frame image is extracted from the video object as the image to be inspected, and a template image corresponding to the image to be inspected is obtained. Before starting the acquisition device for the first time, the user ensures that there are no foreign objects in the video object and selects a frame image without foreign objects from the video stream acquired by the acquisition device, which serves as both the image to be inspected and the template image corresponding to the image to be inspected.

[0032] In this embodiment of the invention, the data acquisition device is used to monitor a preset target area, such as a fire escape route, and to identify foreign objects within the target area using a comparison model. The comparison model requires a template image for identifying foreign objects. In existing technologies, template images are usually fixed and unchanging, which can easily lead to false detections. In this embodiment of the invention, by continuously and smoothly updating the template image, the accuracy of identification can be effectively improved.

[0033] In one embodiment, see Figure 2 , Figure 2 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention. Step S101 above includes steps S201-S202:

[0034] S201, Receives the image selection command input by the user.

[0035] In practice, after the acquisition device is started, it acquires multiple frames of images and provides the user with the option to select one frame from the multiple frames. The image selection command refers to the command issued by the user when selecting a frame.

[0036] S202, according to the image selection instruction, extract the frame image selected by the user from the video object as the image to be inspected, and use the image to be inspected as the template image corresponding to the image to be inspected.

[0037] In specific implementation, a frame image selected by the user is used as both the image to be inspected and the template image corresponding to the image to be inspected. In one embodiment, the image selection command input by the user is only received when the acquisition device is first started, and the frame image selected by the user is determined to be both the image to be inspected and the template image corresponding to the image to be inspected. Subsequently, the preset controller automatically extracts frame images from the preset video object as the image to be inspected and obtains the template image corresponding to the image to be inspected.

[0038] Optionally, a method is provided where users can manually select template images to facilitate adjustments to the selection. Image selection instructions can be specific to user-inputted commands via screen clicks; for example, multiple frame images can be displayed simultaneously on the screen for the user to choose from, with the selected frame image chosen as the image to be inspected.

[0039] S102, based on the template image corresponding to the image to be inspected, determine whether there are foreign objects in the image to be inspected by using a preset comparison model.

[0040] In practice, the comparison model refers to a model that can compare two input images to obtain the differences between the two images. The comparison model can compare the image to be inspected with the template image corresponding to the image to be inspected to analyze whether there are foreign objects in the image to be inspected. If there are foreign objects in the image to be inspected, information such as the coordinates of the foreign objects, the pixel size of the foreign objects, the outline of the foreign objects, and the number of foreign objects can also be obtained.

[0041] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of a sub-process of an automatic template graph updating method provided in an embodiment of the present invention. The comparison model is a Siamese network model, which includes a first network and a second network. Step S102 above includes steps S301-S302:

[0042] In practice, a Siamese neural network, also known as a twin neural network, is a coupled architecture based on two artificial neural networks. A Siamese neural network takes two samples as input, and its two sub-networks (i.e., the first network and the second network) each receive one input and output their representations embedded in a high-dimensional space. The similarity between the two representations is compared by calculating the distance between them, such as Euclidean distance.

[0043] S301, the image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network to obtain the difference result features.

[0044] In specific implementation, the first network and the second network are two sub-networks of the Siamese neural network, each with an input terminal. The image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network. The similarity between the image to be inspected and the template image corresponding to the image to be inspected is compared, and the difference result features between the image to be inspected and the template image corresponding to the image to be inspected are output. The difference result features include foreign object information.

[0045] S302, determine whether there are foreign objects in the image to be inspected based on the characteristics of the difference results.

[0046] In specific implementation, the difference result features are further analyzed to identify foreign object information in the difference result features and determine whether it is a foreign object. In one embodiment, a foreign object filtering model is also included, which can identify and determine whether the foreign object information is a foreign object. If the foreign object information is determined to be a foreign object, it is determined that the image to be inspected contains a foreign object; if the foreign object information is determined not to be a foreign object, the foreign object information included in the difference result features is a false alarm information, and it is determined that the image to be inspected does not contain a foreign object.

[0047] By making full use of twin network technology, we can more accurately identify whether there are foreign objects in the image to be inspected, and prevent the selection of frame images containing foreign objects as template images.

[0048] S103, if there are no foreign objects in the image to be inspected, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected.

[0049] In practice, if the comparison model determines that the image to be inspected does not contain foreign objects, an updated template image can be generated based on the template image corresponding to the image to be inspected, the image to be inspected, and preset weights. Images all include three channels, for example, a 640x640 resolution. Each pixel corresponds to the color values ​​of the three channels, such as (255, 255, 255) for white and (0, 0, 0) for black. The pixel values ​​of the updated template image can be calculated using the pixel values ​​of the template image corresponding to the image to be inspected, the pixel values ​​of the image to be inspected, and the weights.

[0050] In one embodiment, see Figure 5 , Figure 5 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention. Step S103 above includes steps S501-S502:

[0051] S501, respectively obtain the pixel value of the template image corresponding to the image to be inspected and the pixel value of the image to be inspected;

[0052] In practice, an image is obtained by arranging and combining multiple pixel values, where each pixel value is a digital representation of the image. The pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected are obtained respectively.

[0053] S502, the pixel value of the updated template image is calculated based on the pixel value of the template image corresponding to the image to be inspected and the pixel value of the image to be inspected.

[0054] In one embodiment, step S502 includes: using a preset moving average algorithm formula I n =β*I n-1 +(1-β)*I h Obtain the pixel values ​​of the updated template image, where I n-1 I represents the pixel value of the template image corresponding to the image to be inspected. h I represents the pixel value of the image to be inspected. n The pixel values ​​of the updated template image are β, which is a preset weight and is a variable floating-point number between 0 and 1.

[0055] In specific implementation, the weight β ranges from 0 to 1, being a variable floating-point number. A larger weight β means the updated template image is more influenced by the image to be inspected; conversely, a smaller weight β means the updated template image is less affected by the image to be inspected. In one embodiment, a moving average variable is defined. A moving average variable is a type of variable derived from mathematics; it is an abstract concept in computer language that can store calculation results or represent values. The application formula of the variable includes at least two states: before and after the update. The pixel value I of the template image corresponding to the image to be inspected... n-1 , refers to the value of the moving average variable before the update. The pixel value I of the updated template image. n , refers to the value of the updated moving average variable. Typically, the weight β is set to 0.9, even if the resulting updated template image is less affected by the image to be examined.

[0056] The updated template image is adjusted by weight β to measure the degree of influence of the image to be inspected. The value of weight β can be changed by the user, making the resulting updated template image adjustable.

[0057] In one embodiment, step S502 includes: generating a Gaussian mixture model based on the pixel values ​​of the template image corresponding to the image to be inspected; and calculating the pixel values ​​of the updated template image based on the Gaussian mixture model according to the pixel values ​​of the image to be inspected.

[0058] In practice, Gaussian mixture models typically use multiple Gaussian models (usually 3 to 5) to represent the features of each pixel in a frame image. After a new frame is obtained, the Gaussian mixture model is updated. Each pixel in the current image is matched against the Gaussian mixture model; if a match is found, the pixel is considered background; otherwise, it is considered foreground. Foreground refers to any meaningful moving object assuming a static background. The basic idea of ​​modeling is to extract the foreground from the current frame, aiming to make the background more closely resemble the background of the current video frame. This involves updating the background using a weighted average of the current frame and the current background frame in the video sequence. However, due to sudden changes in lighting and other environmental influences, the background after typical modeling is not always clean and clear. Gaussian mixture model (GMM) is one of the most successful modeling methods, and it can also be used for indexing and retrieving surveillance videos. In one embodiment, multiple Gaussian models are established based on all pixel values ​​of the template image corresponding to the image to be tested, and each Gaussian model has corresponding weights. All weighted Gaussian models are combined to obtain a Gaussian mixture model. That is, the pixels of the template image corresponding to the image to be tested are used as the background, and the pixels of the image to be tested are matched with the Gaussian mixture model. If the pixels of the image to be tested match the Gaussian mixture model successfully, then the pixels of the image to be tested are used as the background of the Gaussian mixture model (the background of the new template image); if the pixels of the image to be tested do not match the Gaussian mixture model successfully, then the pixels of the template image corresponding to the image to be tested are still used as the background of the Gaussian mixture model (the background of the new template image), thereby obtaining an updated Gaussian mixture model. After matching all pixels of the image to be tested with the Gaussian mixture model, the image corresponding to the obtained Gaussian mixture model is the updated template image. By obtaining the pixel values ​​of the image to be tested and the moving average and moving variance of the pixel values ​​of the template image corresponding to the image to be tested, the probability that a pixel in the image to be tested belongs to a Gaussian model in the Gaussian mixture model can be calculated. If a pixel in the image to be tested does not belong to any Gaussian model in the Gaussian mixture model, it is determined that the pixel in the image to be tested does not match the Gaussian mixture model successfully, and a Gaussian model with minimal weight is established for that pixel. If a pixel in the image to be tested belongs to a Gaussian model in the Gaussian mixture model, it is determined that the pixel in the image to be tested matches the Gaussian mixture model successfully, and the weight of that Gaussian model is increased.

[0059] By using the Gaussian mixture model, abrupt changes in pixel values ​​can be avoided, resulting in smoother changes in pixel values ​​for each pixel in the image, and consequently, smoother changes in illumination.

[0060] The technical solution of this embodiment, based on the template image corresponding to the image to be inspected, determines whether there are foreign objects in the image to be inspected through a preset comparison model. When there are no foreign objects in the image to be inspected, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected. Updating the template image using an image to be inspected without foreign objects allows the template image to be continuously corrected along with the image to be inspected, making the updated template image more consistent with the current shooting environment and effectively avoiding false detections due to environmental changes. On the one hand, the updated template image reduces the occurrence of false detections; on the other hand, reducing false detections greatly improves the accuracy of subsequent template image updates using the comparison model. The reduction in false detections and the accuracy of template image updates mutually reinforce each other.

[0061] Example 2

[0062] Figure 4 This is a flowchart illustrating an automatic template diagram updating method provided by an embodiment of the present invention. Specifically, this embodiment of the present invention proposes an automatic template diagram updating method; see [link to documentation]. Figure 4 The automatic update method for this template diagram includes the following steps S401-S406.

[0063] S401, extract a frame image from a preset video object as the image to be inspected, and obtain the template image corresponding to the image to be inspected.

[0064] In practice, the video object refers to the video stream or video file acquired by the acquisition device. A frame image is extracted from the video object as the image to be inspected, and a template image corresponding to the image to be inspected is obtained. Before starting the acquisition device for the first time, the user ensures that there are no foreign objects in the video object and selects a frame image without foreign objects from the video stream acquired by the acquisition device, which serves as both the image to be inspected and the template image corresponding to the image to be inspected.

[0065] In this embodiment of the invention, the data acquisition device is used to monitor a preset target area, such as a fire escape route, and to identify foreign objects within the target area using a comparison model. The comparison model requires a template image for identifying foreign objects. In existing technologies, template images are usually fixed and unchanging, which can easily lead to false detections. In this embodiment of the invention, by continuously and smoothly updating the template image, the accuracy of identification can be effectively improved.

[0066] In one embodiment, see Figure 2 , Figure 2 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention. Step S401 above includes steps S201-S202:

[0067] S201, Receives the image selection command input by the user.

[0068] In practice, after the acquisition device is started, it acquires multiple frames of images and provides the user with the option to select one frame from the multiple frames. The image selection command refers to the command issued by the user when selecting a frame.

[0069] S202, according to the image selection instruction, extract the frame image selected by the user from the video object as the image to be inspected, and use the image to be inspected as the template image corresponding to the image to be inspected.

[0070] In practice, a frame image selected by the user is used as both the image to be inspected and the template image corresponding to the image to be inspected.

[0071] Optionally, a method is provided where users can manually select template images to facilitate adjustments to the selection. Image selection instructions can be specific to user-inputted commands via screen clicks; for example, multiple frame images can be displayed simultaneously on the screen for the user to choose from, with the selected frame image chosen as the image to be inspected.

[0072] S402, based on the template image corresponding to the image to be inspected, determine whether there are foreign objects in the image to be inspected by using a preset comparison model.

[0073] In practice, the comparison model refers to a model that can compare two input images to obtain the differences between the two images. The comparison model can compare the image to be inspected with the template image corresponding to the image to be inspected to analyze whether there are foreign objects in the image to be inspected. If there are foreign objects in the image to be inspected, information such as the coordinates of the foreign objects, the pixel size of the foreign objects, the outline of the foreign objects, and the number of foreign objects can also be obtained.

[0074] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of a sub-process of an automatic template graph updating method provided in an embodiment of the present invention. The comparison model is a Siamese network model, which includes a first network and a second network. Step S402 above includes steps S301-S302:

[0075] In practice, a Siamese neural network, also known as a twin neural network, is a coupled architecture based on two artificial neural networks. A Siamese neural network takes two samples as input, and its two sub-networks (i.e., the first network and the second network) each receive one input and output their representations embedded in a high-dimensional space. The similarity between the two representations is compared by calculating the distance between them, such as Euclidean distance.

[0076] S301, the image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network to obtain the difference result features.

[0077] In specific implementation, the first network and the second network are two sub-networks of the Siamese neural network, each with an input terminal. The image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network. The similarity between the image to be inspected and the template image corresponding to the image to be inspected is compared, and the difference result features between the image to be inspected and the template image corresponding to the image to be inspected are output. The difference result features include foreign object information.

[0078] S302, determine whether there are foreign objects in the image to be inspected based on the characteristics of the difference results.

[0079] In specific implementation, the difference result features are further analyzed to identify foreign object information in the difference result features and determine whether it is a foreign object. In one embodiment, a foreign object filtering model is also included, which can identify and determine whether the foreign object information is a foreign object. If the foreign object information is determined to be a foreign object, it is determined that the image to be inspected contains a foreign object; if the foreign object information is determined not to be a foreign object, the foreign object information included in the difference result features is a false alarm information, and it is determined that the image to be inspected does not contain a foreign object.

[0080] By making full use of twin network technology, we can more accurately identify whether there are foreign objects in the image to be inspected, and prevent the selection of frame images containing foreign objects as template images.

[0081] S103, if there are no foreign objects in the image to be inspected, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected.

[0082] In practice, if the comparison model determines that the image to be inspected does not contain foreign objects, an updated template image can be generated based on the template image corresponding to the image to be inspected, the image to be inspected, and preset weights. Images all include three channels, for example, a 640x640 resolution. Each pixel corresponds to the color values ​​of the three channels, such as (255, 255, 255) for white and (0, 0, 0) for black. The pixel values ​​of the updated template image can be calculated using the pixel values ​​of the template image corresponding to the image to be inspected, the pixel values ​​of the image to be inspected, and the weights.

[0083] In one embodiment, see Figure 5 , Figure 5 This is a schematic diagram of a sub-process of an automatic template diagram updating method provided in an embodiment of the present invention. Step S403 above includes steps S501-S502:

[0084] S501, respectively obtain the pixel value of the template image corresponding to the image to be inspected and the pixel value of the image to be inspected;

[0085] In practice, an image is obtained by arranging and combining multiple pixel values, where each pixel value is a digital representation of the image. The pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected are obtained respectively.

[0086] S502, the pixel value of the updated template image is calculated based on the pixel value of the template image corresponding to the image to be inspected and the pixel value of the image to be inspected.

[0087] In one embodiment, step S502 includes: using a preset moving average algorithm formula I n =β*I n-1 +(1-β)*I h Obtain the pixel values ​​of the updated template image, where I n-1 I represents the pixel value of the template image corresponding to the image to be inspected. h I represents the pixel value of the image to be inspected. n The pixel values ​​of the updated template image are β, which is a preset weight and is a variable floating-point number between 0 and 1.

[0088] In specific implementation, the weight β ranges from 0 to 1, being a variable floating-point number. A larger weight β means the updated template image is more influenced by the image to be inspected; conversely, a smaller weight β means the updated template image is less affected by the image to be inspected. In one embodiment, a moving average variable is defined. A moving average variable is a type of variable derived from mathematics; it is an abstract concept in computer language that can store calculation results or represent values. The application formula of the variable includes at least two states: before and after the update. The pixel value I of the template image corresponding to the image to be inspected... n-1 , refers to the value of the moving average variable before the update. The pixel value I of the updated template image. n , refers to the value of the updated moving average variable. Typically, the weight β is set to 0.9, even if the resulting updated template image is less affected by the image to be examined.

[0089] The updated template image is adjusted by weight β to measure the degree of influence of the image to be inspected. The value of weight β can be changed by the user, making the resulting updated template image adjustable.

[0090] In one embodiment, step S502 further includes: generating a Gaussian mixture model based on the pixel values ​​of the template image corresponding to the image to be inspected; and calculating the pixel values ​​of the updated template image based on the Gaussian mixture model according to the pixel values ​​of the image to be inspected.

[0091] In practice, Gaussian mixture models typically use multiple Gaussian models (usually 3 to 5) to represent the features of each pixel in a frame image. After a new frame is obtained, the Gaussian mixture model is updated. Each pixel in the current image is matched against the Gaussian mixture model; if a match is found, the pixel is considered background; otherwise, it is considered foreground. Foreground refers to any meaningful moving object assuming a static background. The basic idea of ​​modeling is to extract the foreground from the current frame, aiming to make the background more closely resemble the background of the current video frame. This involves updating the background using a weighted average of the current frame and the current background frame in the video sequence. However, due to sudden changes in lighting and other environmental influences, the background after typical modeling is not always clean and clear. Gaussian mixture model (GMM) is one of the most successful modeling methods, and it can also be used for indexing and retrieving surveillance videos. In one embodiment, multiple Gaussian models are established based on all pixel values ​​of the template image corresponding to the image to be tested, and each Gaussian model has corresponding weights. All weighted Gaussian models are combined to obtain a Gaussian mixture model. That is, the pixels of the template image corresponding to the image to be tested are used as the background, and the pixels of the image to be tested are matched with the Gaussian mixture model. If the pixels of the image to be tested match the Gaussian mixture model successfully, then the pixels of the image to be tested are used as the background of the Gaussian mixture model (the background of the new template image); if the pixels of the image to be tested do not match the Gaussian mixture model successfully, then the pixels of the template image corresponding to the image to be tested are still used as the background of the Gaussian mixture model (the background of the new template image), thereby obtaining an updated Gaussian mixture model. After matching all pixels of the image to be tested with the Gaussian mixture model, the image corresponding to the obtained Gaussian mixture model is the updated template image. By obtaining the pixel values ​​of the image to be tested and the moving average and moving variance of the pixel values ​​of the template image corresponding to the image to be tested, the probability that a pixel in the image to be tested belongs to a Gaussian model in the Gaussian mixture model can be calculated. If a pixel in the image to be tested does not belong to any Gaussian model in the Gaussian mixture model, it is determined that the pixel in the image to be tested does not match the Gaussian mixture model successfully, and a Gaussian model with minimal weight is established for that pixel. If a pixel in the image to be tested belongs to a Gaussian model in the Gaussian mixture model, it is determined that the pixel in the image to be tested matches the Gaussian mixture model successfully, and the weight of that Gaussian model is increased.

[0092] By using the Gaussian mixture model, abrupt changes in pixel values ​​can be avoided, resulting in smoother changes in pixel values ​​for each pixel in the image, and consequently, smoother changes in illumination.

[0093] S404, the updated template image is used as the template image corresponding to the next frame image of the image to be inspected, and the next frame image of the image to be inspected is extracted from the video object as a new image to be inspected, and then the process proceeds to step S402.

[0094] In specific implementation, the updated template image is used as the template image corresponding to the next frame of the image to be inspected, thereby updating the template image. At the same time, the next frame of the image to be inspected is extracted from the video object as the new image to be inspected, and the above steps S402-S403 are repeated. This enables the template image to be updated according to the image to be inspected as long as there are no foreign objects in the image to be inspected. This allows the template image to be continuously corrected as the image to be inspected is inspected, making the updated template image more consistent with the current shooting environment and effectively avoiding false detections due to changes in the environment.

[0095] S405, if there is a foreign object in the image to be inspected, the template image corresponding to the image to be inspected is used as the template image corresponding to the next frame image of the image to be inspected, and the next frame image of the image to be inspected is extracted from the video object as a new image to be inspected, and then proceed to step S402.

[0096] In practice, if the image to be inspected contains foreign objects, the template image is not updated. This is because updating the template image using the image containing foreign objects would result in the updated template image also containing foreign objects, leading to misjudgments in subsequent identification. Therefore, if the image to be inspected contains foreign objects, the template image corresponding to that image is used as the template image for the next frame of the image to be inspected; that is, the template image for the next frame of the image to be inspected is not updated. Simultaneously, the next frame of the image to be inspected is used as the new image to be inspected, and steps S402-S403 are repeated.

[0097] S406, the automatic template image update method further includes: if there is a foreign object in the image to be inspected, sending the image to be inspected to a preset monitoring terminal.

[0098] In practice, the image of the object to be inspected is sent to a pre-set monitoring terminal. Once the comparison model identifies the object, an alarm is issued in a timely manner, and the user is notified through the monitoring terminal.

[0099] The technical solution of this embodiment, based on the template image corresponding to the image to be inspected, determines whether there are foreign objects in the image to be inspected through a preset comparison model. When there are no foreign objects in the image to be inspected, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected. Updating the template image using an image to be inspected without foreign objects allows the template image to be continuously corrected along with the image to be inspected, making the updated template image more consistent with the current shooting environment and effectively avoiding false detections due to environmental changes. On the one hand, the updated template image reduces the occurrence of false detections; on the other hand, reducing false detections greatly improves the accuracy of subsequent template image updates using the comparison model. The reduction in false detections and the accuracy of template image updates mutually reinforce each other.

[0100] Example 3

[0101] See Figure 6 This invention also provides a template diagram automatic update device 500, which includes a first acquisition unit 501, a first judgment unit 502, and a first calculation unit 503.

[0102] The first acquisition unit 501 is used to extract frame images from a preset video object as images to be inspected, and to acquire the template image corresponding to the images to be inspected.

[0103] In one embodiment, the first acquisition unit 501 specifically includes:

[0104] Receive image selection instructions from the user;

[0105] According to the image selection instruction, the frame image selected by the user is extracted from the video object as the image to be inspected, and the image to be inspected is used as the template image corresponding to the image to be inspected.

[0106] The first judgment unit 502 is used to determine whether there are foreign objects in the image to be inspected based on the template image corresponding to the image to be inspected and through a preset comparison model.

[0107] In one embodiment, the first determination unit 502 specifically includes:

[0108] The image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network to obtain the difference result features;

[0109] Based on the characteristics of the difference results, determine whether there are foreign objects in the image to be inspected.

[0110] The first calculation unit 503 is used to calculate an updated template image based on the template image corresponding to the image to be inspected and the image to be inspected if there are no foreign objects in the image to be inspected.

[0111] In one embodiment, the step of calculating an updated template image based on the template image corresponding to the image to be inspected and the image to be inspected includes:

[0112] The pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected are obtained respectively.

[0113] The pixel values ​​of the updated template image are calculated based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected.

[0114] The step of calculating the pixel values ​​of the updated template image based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected includes:

[0115] Using the preset moving average algorithm formula I n =β*In-1 +(1-β)*I h Obtain the pixel values ​​of the updated template image, where I n-1 I represents the pixel value of the template image corresponding to the image to be inspected. h I represents the pixel value of the image to be inspected. n The pixel values ​​of the updated template image are β, which is a preset weight and ranges from 0 to 1.

[0116] The step of calculating the updated template image based on the template image corresponding to the image to be inspected and the image to be inspected further includes:

[0117] The pixel values ​​of the updated template image are calculated based on the pixel values ​​of the image to be tested and the pixel values ​​of the template image corresponding to the image to be tested, using a preset Gaussian mixture model.

[0118] Example 4

[0119] See Figure 7 This invention also provides a template diagram automatic update device 600, which includes a second acquisition unit 601, a second judgment unit 602, a second calculation unit 603, a first update unit 604, a second update unit 605, and a sending unit 606.

[0120] The second acquisition unit 601 is used to extract frame images from a preset video object as images to be inspected, and to acquire the template image corresponding to the images to be inspected.

[0121] In one embodiment, the second acquisition unit 601 specifically includes:

[0122] Receive image selection instructions from the user;

[0123] According to the image selection instruction, the frame image selected by the user is extracted from the video object as the image to be inspected, and the image to be inspected is used as the template image corresponding to the image to be inspected.

[0124] The second judgment unit 602 is used to determine whether there are foreign objects in the image to be inspected based on the template image corresponding to the image to be inspected and through a preset comparison model.

[0125] In one embodiment, the second determination unit 602 specifically includes:

[0126] The image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network to obtain the difference result features;

[0127] Based on the characteristics of the difference results, determine whether there are foreign objects in the image to be inspected.

[0128] The second calculation unit 603 is used to calculate an updated template image based on the template image corresponding to the image to be inspected and the image to be inspected if there are no foreign objects in the image to be inspected.

[0129] In one embodiment, the step of calculating an updated template image based on the template image corresponding to the image to be inspected and the image to be inspected includes:

[0130] The pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected are obtained respectively.

[0131] The pixel values ​​of the updated template image are calculated based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected.

[0132] The step of calculating the pixel values ​​of the updated template image based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected includes:

[0133] Using the preset moving average algorithm formula I n =β*I n-1 +(1-β)*I h Obtain the pixel values ​​of the updated template image, where I n-1 I represents the pixel value of the template image corresponding to the image to be inspected. h I represents the pixel value of the image to be inspected. n The pixel values ​​of the updated template image are β, which is a preset weight and ranges from 0 to 1.

[0134] The step of calculating the updated template image based on the template image corresponding to the image to be inspected and the image to be inspected further includes:

[0135] The pixel values ​​of the updated template image are calculated based on the pixel values ​​of the image to be tested and the pixel values ​​of the template image corresponding to the image to be tested, using a preset Gaussian mixture model.

[0136] The first update unit 604 is used to use the updated template image as the template image corresponding to the next frame image of the image to be inspected, extract the next frame image of the image to be inspected from the video object as a new image to be inspected, and then transfer to the second judgment unit 602.

[0137] The second updating unit 605 is used to, if there is a foreign object in the image to be inspected, use the template image corresponding to the image to be inspected as the template image corresponding to the next frame image of the image to be inspected, extract the next frame image of the image to be inspected from the video object as a new image to be inspected, and then transfer to the second judgment unit 602.

[0138] The sending unit 606 is used to send the image to be inspected to a preset monitoring terminal if there is a foreign object in the image to be inspected.

[0139] like Figure 8 As shown, this embodiment of the invention provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0140] Memory 113 is used to store computer programs;

[0141] In one embodiment of the present invention, when the processor 111 executes the program stored in the memory 113, it implements the template diagram automatic update method provided in any of the foregoing method embodiments.

[0142] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the template diagram automatic update method provided in any of the foregoing method embodiments.

[0143] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0144] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for automatically updating a template map, characterized by, include: Extract frame images from a preset video object as the image to be inspected, and obtain the template image corresponding to the image to be inspected; Based on the template image corresponding to the image to be inspected, a preset comparison model is used to determine whether there are foreign objects in the image to be inspected. If the image to be inspected does not contain foreign objects, an updated template image is calculated based on the template image corresponding to the image to be inspected and the image to be inspected. The step of calculating the updated template image based on the template image corresponding to the image to be inspected and the image to be inspected includes: The pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected are obtained respectively. The pixel values ​​of the updated template image are calculated based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected. The step of calculating the pixel values ​​of the updated template image based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected includes: A Gaussian mixture model is generated based on the pixel values ​​of the template image corresponding to the image to be inspected; The pixel values ​​of the updated template image are calculated based on the pixel values ​​of the image to be inspected and the Gaussian mixture model.

2. The method of claim 1, wherein, After calculating the updated template image based on the template image corresponding to the image to be inspected and the image to be inspected, the method further includes: The updated template image is used as the template image corresponding to the next frame of the image to be inspected. The next frame of the image to be inspected is extracted from the video object as a new image to be inspected. Then, the process proceeds to the step of determining whether there is a foreign object in the image to be inspected based on the template image corresponding to the image to be inspected and using a preset comparison model.

3. The method of claim 1, wherein, After determining whether there are foreign objects in the image to be inspected based on the template image corresponding to the image to be inspected using a preset comparison model, the method further includes: If the image to be inspected contains foreign objects, the template image corresponding to the image to be inspected is used as the template image corresponding to the next frame image of the image to be inspected. The next frame image of the image to be inspected is extracted from the video object as a new image to be inspected. Then, the process proceeds to the step of determining whether there are foreign objects in the image to be inspected based on the template image corresponding to the image to be inspected and through a preset comparison model.

4. The method of claim 1, wherein, The step of calculating the pixel values ​​of the updated template image based on the pixel values ​​of the template image corresponding to the image to be inspected and the pixel values ​​of the image to be inspected includes: By presetting the sliding average algorithm formula I n = β * I n-1 + (1 - β) * I h Obtain the pixel value of the updated template graph, wherein I n-1 is the pixel value of the template graph corresponding to the to-be-detected graph, I h is the pixel value of the to-be-detected graph, I n is the pixel value of the updated template graph, and β is a preset weight, and the value range of β is 0-1.

5. The method of claim 1, wherein, The comparison model is a Siamese network model, which includes a first network and a second network. The step of determining whether there are foreign objects in the image to be inspected based on the template image corresponding to the image to be inspected, using a preset comparison model, includes: The image to be inspected and the template image corresponding to the image to be inspected are respectively input into the first network and the second network to obtain the difference result features; Based on the characteristics of the difference results, determine whether there are foreign objects in the image to be inspected.

6. A template map automatic updating apparatus characterized by comprising: Includes a unit for performing the method as described in any one of claims 1-5.

7. An electronic device, comprising: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, which when executed by a processor, implements the steps of the method as claimed in any of claims 1-5.

Images