Machine vision event aggregation method and system, computer equipment and medium

Abstract

The invention provides a machine vision event aggregation method and system, computer equipment and a medium, and the method comprises the steps: obtaining an event aggregation strategy which is configured in advance or in real time and is used for aggregating machine vision events, and carrying out the automatic aggregation of the received machine vision events according to the event aggregation strategy, the machine vision event is transmitted through the edge device; and storing and retrieving an automatic aggregation result, exporting a machine vision event image and machine vision event labeling information, and storing and retrieving an execution record of an event aggregation strategy. A user is allowed to configure various event aggregation strategies, and the received machine vision events are automatically aggregated according to the event aggregation strategies, so that aggregation of the machine vision events can be realized based on a multi-strategy rule engine, the receiving problem of a large number of repeated invalid machine vision events is solved, and the user experience is improved. The data processing burden is reduced, and the data processing accuracy and reliability are improved.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine vision and software development, and particularly to a method, system, computer device and medium for aggregating machine vision events. Background Art

[0002] With the extensive development of information technology, machine vision technology has been widely applied in fields such as intelligent parks, intelligent highways, and integrated pipe galleries. However, with the in-depth application, the amount of data to be processed has increased sharply. A large number of repetitive and invalid machine vision events not only increase the burden of data processing, but may also lead to missed reports or delayed responses of key machine vision events, affecting the accuracy and reliability of data processing results. Summary of the Invention

[0003] In view of the above-mentioned disadvantages of the prior art, the purpose of the present invention is to provide a method, system, computer device and medium for aggregating machine vision events, which are used to solve the problem of missed reports or delayed responses of key machine vision events caused by a large number of repetitive machine vision events in the prior art.

[0004] To achieve the above purpose and other related purposes, the present invention provides a method for aggregating machine vision events, including the following steps:

[0005] Obtain an event aggregation strategy configured in advance or in real time for aggregating machine vision events;

[0006] Automatically aggregate the received machine vision events according to the event aggregation strategy, and the machine vision events are transmitted through edge devices;

[0007] Store and retrieve the automatic aggregation results, export machine vision event images and machine vision event annotation information, and store and retrieve the execution records of the event aggregation strategy.

[0008] In an embodiment of the present invention, the process of receiving machine vision events transmitted through edge devices includes:

[0009] Obtain an event receiving interface provided to the edge device in advance or in real time;

[0010] When the edge device calls the event receiving interface through the Hypertext Transfer Protocol or the Hypertext Transfer Security Protocol, transmit the machine vision events in JSON format to a preset topic, and receive the machine vision events transmitted by the edge device by subscribing to the preset topic; and,

[0011] After receiving the machine vision events transmitted by the edge device, parse the machine vision events into a preset data structure.

[0012] In an embodiment of the present invention, the process of automatically aggregating the received machine vision events according to the event aggregation policy includes:

[0013] Based on the machine vision algorithm and the image capturing device, aggregate the machine vision events transmitted within a preset time range into one machine vision event;

[0014] And / or, based on the machine vision algorithm and the image capturing device, perform duplicate image aggregation on the machine vision events transmitted within a preset time range, make a duplicate judgment according to the preset similarity value, add the non-duplicate event images to the valid event library, and add the duplicate event images to the duplicate event library;

[0015] And / or, based on the machine vision algorithm and the image capturing device, perform duplicate image aggregation on each transmitted machine vision event, make a duplicate judgment according to the preset similarity value, add the non-duplicate event images to the valid event library, and add the duplicate event images to the duplicate event library.

[0016] In an embodiment of the present invention, the process of automatically aggregating the received machine vision events according to the event aggregation policy further includes:

[0017] Query the event aggregation policy configuration information from the database based on the machine vision algorithm and the image capturing device;

[0018] Judge whether the transmission time of the machine vision event is within the preset time range through the event aggregation policy configuration information. If it does not exceed the preset time range, add the machine vision event to the duplicate machine vision event library; if it exceeds the preset time range, add the machine vision event to the valid machine vision event library;

[0019] And / or, judge whether the transmission time of the machine vision event is within the preset time range through the event aggregation policy configuration information. If it does not exceed the preset time, add the machine vision event to the interval policy machine vision event delay queue and scan the queue regularly; if the preset time is reached, perform a duplicate image judgment on the machine vision event images transmitted within this time range, and add the duplicate machine vision event images to the duplicate machine vision event library, and add the non-duplicate machine vision event images to the valid machine vision event library; if it exceeds the preset time range, create a new interval policy machine vision event delay queue and wait for the machine vision event to be added;

[0020] And / or, determine whether the machine vision event is the first transmission according to the event aggregation policy configuration information. If the machine vision event is the first transmission, directly add the machine vision event to the valid machine vision event library, and perform duplicate image judgment on the subsequently recognized machine vision events and the image of the most recently transmitted machine vision event based on the image capture device and the machine vision algorithm; if it is judged to be a duplicate, add the newly transmitted machine vision event to the invalid machine vision event library, update the most recent time of the machine vision event, and if it is judged not to be a duplicate, add the machine vision event to the valid machine vision event library and update the most recently transmitted machine vision event.

[0021] In an embodiment of the present invention, the process of performing duplicate image judgment includes:

[0022] Obtain the machine vision event image to be detected;

[0023] Calculate the structural similarity index value of the machine vision event image to be detected, and compare the structural similarity index values of the machine vision event images to be detected;

[0024] When the structural similarity index values of two machine vision event images to be detected exceed the preset similarity value, mark the two machine vision event images to be detected as similar;

[0025] When the structural similarity index values of two machine vision event images to be detected do not exceed the preset similarity value, mark the two machine vision event images to be detected as dissimilar.

[0026] In an embodiment of the present invention, the process of storing and retrieving the automatic aggregation result and exporting the machine vision event image and the machine vision event annotation information includes:

[0027] In response to the automatic aggregation result viewing instruction input by the user, store the automatic aggregation result, and display the machine vision event type, image capture device, occurrence time, machine vision event image, and machine vision event valid status in a list manner and / or a tiled manner; and,

[0028] In response to the automatic aggregation result retrieval instruction input by the user, page query and retrieve the machine vision events by machine vision event type, geographical location, and occurrence time, and export the images and annotation information of the machine vision events selected by the user in the form of a compressed package.

[0029] In an embodiment of the present invention, the process of storing and retrieving the execution record of the event aggregation policy includes:

[0030] In response to the execution record viewing instruction input by the user, store the execution record of the event aggregation policy, and display the policy name, policy type, execution time, and execution status; and,

[0031] In response to an execution record retrieval instruction input by the user, retrieve the execution records of the event aggregation policy by paging query according to the policy name, policy type, and execution time, and display the images, occurrence times, and similarities of the aggregated machine vision events in the form of cards.

[0032] The present invention also provides a machine vision event aggregation system, which includes:

[0033] A data receiving module, configured to receive machine vision events transmitted through edge devices;

[0034] A rule engine module, configured to obtain event aggregation policies for aggregating machine vision events configured in advance or in real time, and automatically aggregate the received machine vision events according to the event aggregation policies;

[0035] An event management module, configured to store and retrieve the automatic aggregation results, and export machine vision event images and machine vision event annotation information;

[0036] A rule log module, configured to store and retrieve the execution records of the event aggregation policies.

[0037] The present invention also provides a computer device, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the machine vision event aggregation method described in any one of the above.

[0038] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the machine vision event aggregation method described in any one of the above are implemented.

[0039] As described above, the present invention provides a method, system, computer device and medium for aggregating machine vision events, which have the following beneficial effects: By obtaining an event aggregation strategy configured in advance or in real time for aggregating machine vision events, and automatically aggregating the received machine vision events according to the event aggregation strategy, the machine vision events are transmitted through edge devices; then, the automatic aggregation results are stored and retrieved, and the machine vision event images and machine vision event annotation information are exported, as well as the execution records of the event aggregation strategy are stored and retrieved. It can be seen from this that the present invention allows users to configure multiple event aggregation strategies and automatically aggregate the received machine vision events according to these event aggregation strategies, so that the present invention can realize the aggregation of machine vision events based on a multi-strategy rule engine, solve the problem of receiving a large number of repetitive and invalid machine vision events, reduce the data processing burden, and improve the accuracy and reliability of data processing. At the same time, the present invention can store the aggregated machine vision events, provide viewing and searching functions for the aggregated machine vision events, and support the export of machine vision event images and annotation information; moreover, the present invention can also record the execution situation of the rule engine and the results of machine vision event aggregation, which is convenient for subsequent viewing and retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of an exemplary system architecture for applying the technical solutions in one or more embodiments of the present invention;

[0041] Figure 2 Flow chart of the method for aggregating machine vision events provided in an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of the principle of the system for aggregating machine vision events provided in an embodiment of the present invention;

[0043] Figure 4 Timing diagram of the system for aggregating machine vision events provided in an embodiment of the present invention;

[0044] Figure 5 Hardware structure diagram of a computer device suitable for implementing one or more embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It can be understood that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. In addition, it can be understood that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0046] Figure 1 FIG. shows a schematic diagram of an exemplary system architecture to which the technical solutions in one or more embodiments of the present invention can be applied. As Figure 1 shown, the system architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as a smart phone, a tablet computer, a laptop computer, and a desktop computer. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types capable of providing a communication link between the terminal device 110 and the server 130. For example, it may be a wired communication link or a wireless communication link.

[0047] According to the implementation requirements, the system architecture in the embodiments of the present invention may have any number of terminal devices, networks, and servers. For example, the server 130 may be a server group composed of multiple server devices. In addition, the technical solutions provided in the embodiments of the present invention can be applied to the terminal device 110, or can be applied to the server 130, or can be jointly implemented by the terminal device 110 and the server 130. The present invention makes no special limitation on this.

[0048] In an embodiment of the present invention, the terminal device 110 or the server 130 of the present invention can obtain an event aggregation policy configured in advance or in real time for aggregating machine vision events, and automatically aggregate the received machine vision events according to the event aggregation policy. The machine vision events are transmitted through edge devices; then, the automatic aggregation result is stored and retrieved, and the machine vision event images and the machine vision event annotation information are exported, and the execution records of the event aggregation policy are stored and retrieved. By using the terminal device 110 or the server 130 to execute the machine vision event aggregation method, users can be allowed to configure multiple event aggregation policies and automatically aggregate the received machine vision events according to these event aggregation policies, so that the aggregation of machine vision events can be realized based on a multi-policy rule engine, the problem of receiving a large number of repeated and invalid machine vision events can be solved, the data processing burden can be reduced, and the accuracy and reliability of data processing can be improved. At the same time, the aggregated machine vision events can be stored, the viewing and searching functions of the aggregated machine vision events can be provided, and the export of the machine vision event images and annotation information is supported; moreover, the execution situation of the rule engine and the result of the machine vision event aggregation can be recorded, which is convenient for subsequent viewing and retrieval.

[0049] The above part introduced the content of the exemplary system architecture applying the technical solution of the present invention. Next, the machine vision event aggregation method of the present invention will be continued to be introduced.

[0050] Among them, a machine vision event can be an event that analyzes and processes images or videos in fields such as smart campuses, smart highways, and integrated pipe galleries through machine vision technology to detect and identify specific behaviors or targets in these images or videos.

[0051] Figure 2 Shows a flowchart of a machine vision event aggregation method. Specifically, in an exemplary embodiment, as Figure 2 shown, this embodiment provides a machine vision event aggregation method, including the following steps:

[0052] S210, obtaining an event aggregation policy configured in advance or in real time for aggregating machine vision events;

[0053] S220, automatically aggregating the received machine vision events according to the event aggregation policy, and the machine vision events are transmitted through edge devices;

[0054] S230, storing and retrieving the automatic aggregation result, exporting the machine vision event images and the machine vision event annotation information, and storing and retrieving the execution records of the event aggregation policy.

[0055] According to the above description, in an exemplary embodiment, the process of receiving machine vision events transmitted by an edge device may include: obtaining an event receiving interface provided to the edge device in advance or in real time; when the edge device calls the event receiving interface through the Hyper Text Transfer Protocol (HTTP) or the Hypertext Transfer Protocol Secure (HTTPS), transmitting the machine vision events in JSON format to a preset topic, and receiving the machine vision events transmitted by the edge device by subscribing to the preset topic; and, after receiving the machine vision events transmitted by the edge device, parsing the machine vision events into a preset data structure.

[0056] According to the above description, in an exemplary embodiment, the process of automatically aggregating the received machine vision events according to an event aggregation strategy may include: aggregating the machine vision events transmitted within a preset time range into one machine vision event based on a machine vision algorithm and an image capturing device. As another example, the process of automatically aggregating the received machine vision events according to an event aggregation strategy may include: performing repeated image aggregation on the machine vision events transmitted within a preset time range based on a machine vision algorithm and an image capturing device, making a duplicate judgment according to a preset similarity value, adding the non-duplicate event images to an effective event library, and adding the duplicate event images to a duplicate event library. As yet another example, based on a machine vision algorithm and an image capturing device, performing repeated image aggregation on each transmitted machine vision event, making a duplicate judgment according to a preset similarity value, adding the non-duplicate event images to an effective event library, and adding the duplicate event images to a duplicate event library.

[0057] According to the above description, in an exemplary embodiment, the process of automatically aggregating received machine vision events according to an event aggregation policy further includes: querying event aggregation policy configuration information from a database based on a machine vision algorithm and an image capturing device, and judging whether the transmission time of the machine vision event is within a preset time range through the event aggregation policy configuration information. If it does not exceed the preset time range, the machine vision event is added to the repeated machine vision event library; if it exceeds the preset time range, the machine vision event is added to the valid machine vision event library. As another example, the process of automatically aggregating received machine vision events according to an event aggregation policy further includes: judging whether the transmission time of the machine vision event is within a preset time range through the event aggregation policy configuration information. If it does not exceed the preset time, the machine vision event is added to the interval policy machine vision event delay queue and the queue is scanned regularly; if the preset time is reached, the machine vision event images transmitted within this time range are judged for duplicate images, and the duplicate machine vision event images are added to the repeated machine vision event library, and the non-duplicate machine vision event images are added to the valid machine vision event library; if it exceeds the preset time range, a new interval policy machine vision event delay queue is created and waiting for machine vision events to be added. As yet another example, it is judged through the event aggregation policy configuration information whether the machine vision event is the first transmission. If the machine vision event is the first transmission, the machine vision event is directly added to the valid machine vision event library, and subsequent recognized machine vision events are judged for duplicate images with the most recently transmitted machine vision event images based on the image capturing device and the machine vision algorithm; if it is judged to be duplicate, the newly transmitted machine vision event is added to the invalid machine vision event library and the most recent time of the machine vision event is updated. If it is judged not to be duplicate, the machine vision event is added to the valid machine vision event library and the most recently transmitted machine vision event is updated.

[0058] According to the above description, in an exemplary embodiment, the process of performing duplicate image determination includes: obtaining a machine vision event image to be detected; calculating the structural similarity index value of the machine vision event image to be detected and comparing the structural similarity index value of the machine vision event image to be detected; when the structural similarity index values of two machine vision event images to be detected exceed a preset similarity value, marking the two machine vision event images to be detected as similar; when the structural similarity index values of two machine vision event images to be detected do not exceed the preset similarity value, marking the two machine vision event images to be detected as dissimilar. Specifically, the process of calculating the structural similarity index value of the machine vision event image to be detected may include: calculating the brightness, contrast, and structural similarity of the two images respectively; calculating the structural similarity index (Structure Similarity Index Measure, abbreviated as SSIM) value according to the brightness, contrast, and structural similarity, where the SSIM value calculation formula is: SSIM(x, y) = (2μ_xμ_y + C1) * (2σ_xy + C2) / (μ_x^2 + μ_y^2 + C1) * (σ_x^2 + σ_y^2 + C2), where μ_x represents the average brightness of image x, μ_y represents the average brightness of image y, σ_x^2 represents the variance of image x, σ_y^2 represents the variance of image y, σ_xy represents the covariance of image x and image y, and C1 and C2 are constants used to avoid the denominator being zero.

[0059] According to the above description, in an exemplary embodiment, the process of storing and retrieving the automatic aggregation result and exporting the machine vision event image and the machine vision event annotation information includes: in response to an automatic aggregation result viewing instruction input by the user, storing the automatic aggregation result and displaying the machine vision event type, image capturing device, occurrence time, machine vision event image, and machine vision event valid state in a list manner and / or a tiled manner; and, in response to an automatic aggregation result retrieval instruction input by the user, paging and querying and retrieving machine vision events by machine vision event type, geographical location, and occurrence time, and exporting the images and annotation information of the machine vision events selected by the user in a compressed package form.

[0060] According to the above description, in an exemplary embodiment, the process of storing and retrieving the execution record of the event aggregation policy includes: in response to an execution record viewing instruction input by the user, storing the execution record of the event aggregation policy and displaying the policy name, policy type, execution time, and execution status; and, in response to an execution record retrieval instruction input by the user, paging and querying and retrieving the execution record of the event aggregation policy by policy name, policy type, and execution time, and displaying the image, occurrence time, and similarity of the aggregated machine vision events in a card form.

[0061] In another exemplary embodiment of the present invention, this embodiment further provides a machine vision event aggregation method, including the following steps:

[0062] The user configures the machine vision event aggregation rules in the rule engine module. Specifically, the machine vision event aggregation rules configured by the user in the rule engine include: 1) Time dimension aggregation strategy: For the selected machine vision algorithm and image capture device, the machine vision events transmitted within a preset time range are aggregated into one machine vision event. 2) Interval duplicate image aggregation strategy: For the selected machine vision algorithm and image capture device, for the machine vision events transmitted within a preset time range, duplicate image aggregation is performed, and duplicate judgment is made according to the set similarity. The machine vision events judged not to be duplicates are added to the valid machine vision event library, and the duplicate images are added to the duplicate machine vision event library. 3) Real-time duplicate image aggregation strategy: For the selected machine vision algorithm and image capture device, for each transmitted machine vision event, duplicate image aggregation is performed, and duplicate judgment is made according to the set similarity. The machine vision events judged not to be duplicates are added to the valid machine vision event library, and the duplicate images are added to the duplicate machine vision event library.

[0063] The data receiving module receives machine vision events. Specifically, the data receiving module receives the machine vision events transmitted by the edge device, and the receiving methods include: 1) HTTP / HTTPS method: Provide a machine vision event receiving interface, and the edge device can transmit the machine vision events in JSON format by calling the interface through HTTP / HTTPS; 2) Message queue (MQ) subscription method: The edge device pushes the generated machine vision events in JSON format to the specified topic, and receives the machine vision events by subscribing to this topic; after receiving the machine vision events, the machine vision events are uniformly parsed into the required data structure.

[0064] The rule engine module automatically aggregates machine vision events according to the configured policies. Specifically, the process of the rule engine module automatically aggregating machine vision events according to the configured policies includes: 1) Based on the parsed machine vision events, query the aggregation policy configuration from the database according to the image capture device and algorithm information; 2) Determine the policy type and execute the aggregation method: 2-a) Time dimension aggregation policy: Determine whether the transmission time of the machine vision event is within the preset time range. If it does not exceed the preset time range, add the machine vision event to the repeated machine vision event library; if it exceeds the preset time range, add the machine vision event to the valid machine vision event library; 2-b) Interval repeated image aggregation policy: Determine whether the transmission time of the machine vision event is within the preset time range. If it does not exceed the preset time, add the machine vision event to the interval policy machine vision event delay queue, set a timer to scan the queue regularly. If the preset time is reached, perform a repeated image judgment on the machine vision event images transmitted within this time range. Add the determined repeated machine vision events to the repeated machine vision event library, and add the determined non-repeated machine vision events to the valid machine vision event library; if it exceeds the preset time range, create a new interval policy machine vision event delay queue and wait for machine vision events to be added; 2-c) Real-time repeated image aggregation policy: If the machine vision event is transmitted for the first time, directly add it to the valid machine vision event library; for subsequent machine vision events of this algorithm recognized by this image capture device, perform a repeated image judgment with the most recently transmitted machine vision event image. If it is judged to be repeated, add the newly transmitted machine vision event to the invalid machine vision event library and update the most recent time of the machine vision event. If it is judged not to be repeated, add the machine vision event to the valid machine vision event library and update the most recently transmitted machine vision event. Among them, regarding the use of the SSIM algorithm for the similarity index of the repeated image judgment interface structure: 1) Input the machine vision event image to be detected and the preset threshold; 2) Process the input image with the SSIM algorithm to obtain their respective SSIM values; 3) Taking the first image as the reference, compare the SSIM values of the images. When the SSIM value exceeds the preset threshold, judge the two images as repeated images.The process of processing the input image using the SSIM algorithm may include: calculating the brightness, contrast, and structural similarity of two images respectively; calculating the Structure Similarity Index Measure (SSIM) value based on the brightness, contrast, and structural similarity. The formula for the SSIM value is: SSIM(x, y) = (2μ_xμ_y + C1) * (2σ_xy + C2) / (μ_x^2 + μ_y^2 + C1) * (σ_x^2 + σ_y^2 + C2), where μ_x represents the average brightness of image x, μ_y represents the average brightness of image y, σ_x^2 represents the variance of image x, σ_y^2 represents the variance of image y, σ_xy represents the covariance between image x and image y, and C1 and C2 are constants used to avoid a zero denominator.

[0065] The event management module stores and retrieves the aggregation results and provides functions for exporting machine vision event images and machine vision event annotations. Specifically, users can view the aggregated machine vision events, which support displaying information such as the machine vision event type, image capture device, occurrence time, machine vision event image, and machine vision event valid status in two ways: list / tile; and support paging query of machine vision events by machine vision event type, geographical location, and occurrence time. Users can select the required machine vision events and export the images and annotation information of the machine vision events in the form of a compressed package.

[0066] The rule log module stores and retrieves the execution records of the rule engine and the aggregation status of machine vision events. Specifically, users can view the execution records of the rule engine, which display information such as the policy name, policy type, execution time, and execution status; and support paging query of records by policy name, policy type, and execution time. Each rule engine execution record provides a details operation, through which the details of the machine vision events for the executed policy under this record can be viewed, and information such as the image, occurrence time, and similarity of the aggregated machine vision events is displayed in the form of a card.

[0067] In summary, the present invention provides a method for aggregating machine vision events. By obtaining an event aggregation strategy configured in advance or in real time for aggregating machine vision events, and automatically aggregating the received machine vision events according to the event aggregation strategy, the machine vision events are transmitted through edge devices; then, the automatic aggregation results are stored and retrieved, and the machine vision event images and machine vision event annotation information are exported, as well as the execution records of the event aggregation strategy are stored and retrieved. It can be seen from this that the present invention allows users to configure multiple event aggregation strategies, and automatically aggregate the received machine vision events according to these event aggregation strategies, so that the present invention can realize the aggregation of machine vision events based on a multi-strategy rule engine, solve the problem of receiving a large number of repetitive and invalid machine vision events, reduce the data processing burden, and improve the accuracy and reliability of data processing. At the same time, the present invention can store the aggregated machine vision events, provide viewing and searching functions for the aggregated machine vision events, and support the export of machine vision event images and annotation information; moreover, the present invention can also record the execution situation of the rule engine and the results of machine vision event aggregation, which is convenient for subsequent viewing and retrieval.

[0068] In another exemplary embodiment of the present invention, as Figure 3 shown, this embodiment further provides a machine vision event aggregation system, including:

[0069] A data receiving module, configured to receive machine vision events transmitted through edge devices. Among them, the data receiving module can receive the machine vision events transmitted by the edge device through two methods: HTTP / HTTPS and MQ subscription, and uniformly parse the machine vision events into the data structure required by the system.

[0070] A rule engine module, configured to obtain an event aggregation strategy configured in advance or in real time for aggregating machine vision events, and automatically aggregate the received machine vision events according to the event aggregation strategy. Among them, the rule engine module allows users to configure multiple event aggregation strategies, and automatically aggregate the received machine vision events according to these event aggregation strategies. At the same time, the rule engine module can provide various strategies for aggregating machine vision events, such as time dimension aggregation, interval repeated image aggregation, and real-time repeated image aggregation. The rule engine automatically aggregates and processes the machine vision events based on the configured strategy.

[0071] An event management module, configured to store and retrieve the automatic aggregation results, and export the machine vision event images and machine vision event annotation information. Among them, the event management module can store the aggregated machine vision events, provide viewing and searching functions for the aggregated machine vision events, and support the export of machine vision event images and annotation information.

[0072] A rule log module for storing and retrieving execution records of event aggregation policies. Among them, the rule log module can record the execution status of the rule engine and the results of machine vision event aggregation, facilitating subsequent viewing and retrieval.

[0073] According to the above description, in an exemplary embodiment, as Figure 4 shown, the user can configure different machine vision event aggregation policies for each algorithm according to the actual situation of the project in the rule engine module, including time dimension aggregation, interval duplicate image aggregation, and real-time duplicate image aggregation. After the data receiving module receives the machine vision events from the edge device, it uniformly parses them into the data structure required by the system, and then the rule engine queries the corresponding policy according to the device and algorithm information of the machine vision event, and performs aggregation processing on the machine vision event. The aggregated machine vision events are stored and retrieved by the event management module, which supports the export of machine vision event images and machine vision event annotation information for subsequent vision processing. The rule log module stores and retrieves the execution records of the rule engine and the aggregation status of machine vision events.

[0074] According to the above description, in an exemplary embodiment, the hardware environment used by the machine vision event aggregation system is CentOS X86_64, 16-core CPU, 32GB of memory; the development language of the machine vision event aggregation system is Java8, and it is built using the SpringBoot2.2.13 framework; MySQL 8 is used to store and manage machine vision event aggregation policies, and Elasticsearch7.17.6 is used to store and retrieve a large number of visual machine vision events.

[0075] According to the above description, as an example, by simulating the operation environment of an intelligent park, the edge device transmitted real-time image data of machine vision events of multiple visual algorithms including vehicle illegal parking, personnel leaving the post, perimeter intrusion, and garbage detection, with a total of more than 20,000 pieces. Through the efficient aggregation policy configured by the rule engine adopted in the above embodiment, more than 300 effective machine vision events can be quickly processed and aggregated within 3 to 5 seconds.

[0076] It can be understood that the machine vision event aggregation system provided in the above embodiment and the machine vision event aggregation method provided in the above embodiment belong to the same concept. The specific manner of performing operations in the machine vision event aggregation method has been described in detail in the above method embodiment, and will not be repeated here. In actual application, the machine vision event aggregation system provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the machine vision event aggregation system into different functional modules, and then implement all or part of the functions of the corresponding functional modules through the machine vision event aggregation method described in the above embodiment. No specific limitation is imposed here either.

[0077] In summary, the present invention provides a machine vision event aggregation system. The system receives machine vision events transmitted through edge devices via a data reception module; obtains event aggregation policies configured in advance or in real time for aggregating machine vision events through a rule engine module, and automatically aggregates the received machine vision events according to the event aggregation policies; stores and retrieves the automatic aggregation results through an event management module, and exports machine vision event images and machine vision event annotation information; stores and retrieves the execution records of the event aggregation policies through a rule log module. It can be seen from this that the present invention allows users to configure multiple event aggregation policies, and automatically aggregates the received machine vision events according to these event aggregation policies, enabling the present invention to achieve the aggregation of machine vision events based on a multi-policy rule engine, solving the problem of receiving a large number of repetitive and ineffective machine vision events, reducing the data processing burden, and improving the accuracy and reliability of data processing. At the same time, the present invention can store the aggregated machine vision events, provide viewing and searching functions for the aggregated machine vision events, and support the export of machine vision event images and annotation information; moreover, the present invention can also record the execution situation of the rule engine and the results of machine vision event aggregation, facilitating subsequent viewing and retrieval.

[0078] An embodiment of the present invention further provides a computer device, which may include a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to cause the computer device to execute Figure 2 the steps of the machine vision event aggregation method described above. Figure 5 The structural schematic diagram of a computer device 1000 is shown. Refer to Figure 5 As shown, the computer device 1000 includes: a processor 1010, a memory 1020, a power supply 1030, a display unit 1040, and an input unit 1060.

[0079] The processor 1010 is the control center of the computer device 1000, connects various components using various interfaces and lines, and executes various functions of the computer device 1000 by running or executing computer programs / instructions stored in the memory 1020, thereby monitoring the computer device 1000 as a whole. In the embodiment of the present invention, when the processor 1010 calls the computer program stored in the memory 1020, it executes as Figure 2Steps of the machine vision event aggregation method described above. Optionally, the processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, applications, etc., and the modem processor mainly processes wireless communication. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be separately implemented on independent chips.

[0080] The memory 1020 may mainly include a program storage area and a data storage area. Among them, the program storage area may store the operating system, various applications, etc.; the data storage area may store instruction data created according to the use of the computer device 1000, etc. In addition, the memory 1020 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices, etc.

[0081] The computer device 1000 further includes a power supply 1030 (such as a battery) for powering each component. The power supply may be logically connected to the processor 1010 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption through the power management system.

[0082] The display unit 1040 may be used to display information input by the user or information provided to the user, as well as various menus of the computer device 1000, etc. In the embodiments of the present invention, it is mainly used to display the display interfaces of various applications in the computer device 1000 and objects such as text and pictures displayed in the display interfaces. The display unit 1040 may include a display panel 1050. The display panel 1050 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0083] The input unit 1060 may be used to receive information such as numbers or characters input by the user. The input unit 1060 may include a touch panel 1070 and other input devices 1080. Among them, the touch panel 1070, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 1070).

[0084] Specifically, the touch panel 1070 can detect a user's touch operation, detect the signals brought about by the touch operation, convert these signals into contact coordinates, send them to the processor 1010, and receive and execute the commands sent by the processor 1010. In addition, the touch panel 1070 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave. The other input devices 1080 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), trackball, mouse, joystick, etc.

[0085] Of course, the touch panel 1070 can cover the display panel 1050. After the touch panel 1070 detects a touch operation on or near it, it is transmitted to the processor 1010 to determine the type of touch event. Subsequently, the processor 1010 provides a corresponding visual output on the display panel 1050 according to the type of touch event. Although in Figure 5 the touch panel 1070 and the display panel 1050 are implemented as two independent components to realize the input and output functions of the computer device 1000, in some embodiments, the touch panel 1070 and the display panel 1050 can be integrated to realize the input and output functions of the computer device 1000.

[0086] The computer device 1000 may further include one or more sensors, such as a pressure sensor, a gravitational acceleration sensor, a proximity light sensor, etc. Of course, according to the needs in specific applications, the above computer device 1000 may further include other components such as a camera.

[0087] The embodiment of the present invention also provides a computer-readable storage medium. A computer program / instructions is stored in the storage medium. When the computer program / instructions is executed by a processor, the above device can execute the steps of the machine vision event aggregation method as described in Figure 2 the present invention.

[0088] Those skilled in the art can understand that Figure 5 merely examples of computer devices are given, which do not constitute a limitation to the device. The device may include more or fewer components than shown in the figure, or combine some components, or different components. For the convenience of description, the above parts are divided into various modules (or units) according to functions and described separately. Of course, when implementing the present invention, the functions of the various modules (or units) can be implemented in the same or multiple software or hardware.

[0089] Those skilled in the art should understand that the present invention can be implemented in the form of a computer program product on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be applied to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0090] It can be understood that when the above embodiments perform processing such as collecting, storing, using, processing, transmitting, providing, disclosing, deleting, etc. on relevant data (such as machine vision images or machine vision image data, etc.), it is completed with the consent of the user or after obtaining the consent of the user. For example, machine vision images or machine vision image data are obtained by authorization with the knowledge and consent of the user; or are actively provided by the user after reading the relevant instructions, or are actively authorized / provided / transmitted by the user when using some or all of the functions described in the above embodiments, or are obtained through other means / ways with the consent of the user or after obtaining the consent of the user.

[0091] The above embodiments are only illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for aggregating machine vision events, characterized in that: The method comprises: Obtain pre-configured or real-time event aggregation strategies for aggregating machine vision events; Automatically aggregating received machine vision events according to the event aggregation strategy, wherein the machine vision events are transmitted through edge devices; The automatic aggregation results are stored and retrieved, and the machine vision event images and machine vision event annotation information are exported, and the execution records of the event aggregation strategy are stored and retrieved.

2. The machine vision event aggregation method according to claim 1, characterized in that: The process of receiving machine vision events transmitted by edge devices includes: Acquire an event receiving interface provided to the edge device in advance or in real time; When the edge device calls the event receiving interface through the Hypertext Transfer Protocol or the Hypertext Transfer Protocol Secure, the machine vision event is transmitted to a preset topic in the form of JSON, and the machine vision event transmitted by the edge device is received by subscribing to the preset topic; and After receiving the machine vision event transmitted by the edge device, the machine vision event is parsed into a preset data structure.

3. The machine vision event aggregation method according to claim 1 or 2, characterized in that: The process of automatically aggregating the received machine vision events according to the event aggregation strategy includes: Based on the machine vision algorithm and the image capturing device, the machine vision events transmitted within a preset time range are aggregated into one machine vision event; and / or, based on a machine vision algorithm and an image capturing device, performing repeated image aggregation on machine vision events transmitted within a preset time range, and performing duplication judgment according to a preset similarity value, adding non-duplicate event images to a valid event library, and adding duplicate event images to a duplicate event library; And / or, based on the machine vision algorithm and the image capture device, repeated images are aggregated for each transmitted machine vision event, and repeated judgments are made according to preset similarity values, and non-repeated event images are added to the valid event library, and repeated event images are added to the repeated event library.

4. The machine vision event aggregation method according to claim 3, characterized in that: The process of automatically aggregating the received machine vision events according to the event aggregation strategy also includes: Querying event aggregation strategy configuration information from a database based on a machine vision algorithm and an image capture device; Determine whether the machine vision event transmission time is within a preset time range through the event aggregation strategy configuration information; if it does not exceed the preset time range, add the machine vision event to a repeated machine vision event library; if it exceeds the preset time range, add the machine vision event to a valid machine vision event library; And / or, judging whether the machine vision event transmission time is within a preset time range through the event aggregation strategy configuration information; if it does not exceed the preset time, adding the machine vision event to the interval strategy machine vision event delay queue, and scanning the queue regularly; if the preset time is reached, performing duplicate image judgment on the machine vision event images transmitted within the time range, and adding the duplicate machine vision event images to the duplicate machine vision event library, and adding the non-duplicate machine vision event images to the valid machine vision event library; if it exceeds the preset time range, creating a new interval strategy machine vision event delay queue, and waiting for the machine vision event to be added; And / or, determine whether the machine vision event is transmitted for the first time through the event aggregation strategy configuration information; if the machine vision event is transmitted for the first time, directly add the machine vision event to the valid machine vision event library, and perform duplicate image judgment on the subsequently identified machine vision event and the most recently transmitted machine vision event image based on the image capture device and the machine vision algorithm; if it is judged to be repeated, add the newly transmitted machine vision event to the invalid machine vision event library, and update the most recent time of the machine vision event; if it is judged to be not repeated, add the machine vision event to the valid machine vision event library, and update the most recently transmitted machine vision event.

5. The machine vision event aggregation method according to claim 4, characterized in that: The process of performing repeated image judgment includes: Acquire a machine vision event image to be detected; Calculating the structural similarity index value of the machine vision event image to be detected, and comparing it with the structural similarity index value of the machine vision event image to be detected; When the structural similarity index value of two machine vision event images to be detected exceeds a preset similarity value, the two machine vision event images to be detected are marked as similar; When the structural similarity index value of two machine vision event images to be detected does not exceed a preset similarity value, the two machine vision event images to be detected are marked as dissimilar.

6. The machine vision event aggregation method according to claim 1, characterized in that: The process of storing and retrieving the automatic aggregation results and exporting the machine vision event images and machine vision event annotation information includes: In response to an automatic aggregation result viewing instruction input by a user, the automatic aggregation results are stored, and the machine vision event type, image capturing device, occurrence time, machine vision event image, and machine vision event validity status are displayed in a list and / or tiled manner; and, In response to the automatic aggregation result retrieval instruction input by the user, the machine vision events are retrieved by paging query based on the machine vision event type, geographic location, and occurrence time, and the images and annotation information of the machine vision events selected by the user are exported in the form of a compressed package.

7. The machine vision event aggregation method according to claim 1, characterized in that: The process of storing and retrieving the execution records of the event aggregation strategy includes: In response to an execution record viewing instruction input by a user, the execution record of the event aggregation strategy is stored, and the strategy name, strategy type, execution time and execution status are displayed; and, In response to the execution record retrieval instruction input by the user, the execution record of the event aggregation strategy is retrieved by paging query based on the strategy name, strategy type and execution time, and the image, occurrence time and similarity of the aggregated machine vision event are displayed in the form of cards.

8. A machine vision event aggregation system, characterized in that: The system comprises: A data receiving module, used for receiving machine vision events transmitted through edge devices; A rule engine module, configured to obtain a pre-configured or real-time event aggregation strategy for aggregating machine vision events, and automatically aggregate received machine vision events according to the event aggregation strategy; An event management module is used to store and retrieve the automatic aggregation results and export machine vision event images and machine vision event annotation information; The rule log module is used to store and retrieve the execution records of the event aggregation strategy.

9. A computer device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the machine vision event aggregation method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the machine vision event aggregation method described in any one of claims 1 to 7 are implemented.

Images