Systems and methods for adaptively organized process parallelization

Abstract

A method of process parallelization includes receiving an insight request from one of a plurality of client systems of the process parallelization system, analyzing the received request to identify objects, operators, modifiers, and links therein, identifying sensors and locations required to capture measurement variables corresponding to the identified objects, selecting the identified sensors from a bank of available sensors, capturing data acquired by the selected sensors, identifying classifiers that can be used to detect the identified objects, selecting the identified classifiers from a bank of available object and activity classifiers, processing the data captured by the selected sensors with the selected classifiers, configuring expert system rules according to the identified objects, operators, modifiers, and links, processing output from the selected classifiers with the expert system rules, compiling an insight data packet (IDP) with output of the selected classifiers, and distributing the IDP to the client systems.

Description

[Technical Field]

[0001] The present disclosure relates generally to systems and methods that enable multiple users in parallel to create / compose their own high-level queries about an observed scene. More particularly, the present disclosure relates to systems and methods that enable multiple users in parallel to create / compose their own high-level queries about an observed scene and receive corresponding high-level results from an artificial intelligence (AI) system using data from user-selectable or automatically selected sensors according to a semantic analysis of the high-level queries. [Background technology]

[0002] The current approach to the development of AI-based systems is marked by a reductionist mindset, where point solutions are developed for individual use-case scenarios and supplemented by hard-coding as needed. Keeping in mind the expanding use of AI-based technologies for a wide range of applications, large numbers of highly skilled AI experts are needed to develop the required AI solutions. However, there is a widening gap between the pipeline of available AI experts and the demand for AI solutions. Summary of the Invention

[0003] In one aspect of the present disclosure, a process parallelization system is provided. The process parallelization system includes a core processor communicatively coupled to a plurality of sensors and a data packet distribution system coupled to a plurality of client systems via corresponding two-way application programming interfaces (APIs). The core processor includes a query processor configured to receive an insight request from one of the plurality of client systems. The query processor includes a semantic analyzer configured to analyze the received insight request to identify subjects, operators, modifiers, and links therein, a sensor identifier unit configured to identify sensors and corresponding locations required to capture measurement variables corresponding to the identified subjects, and a rule formulator unit configured to generate one or more rules based on the received insight request. Moreover, the core processor also includes a sensor selector unit configured to select the identified sensor from the plurality of sensors for capturing data directly related to the identified subject.Additionally, the core processor also includes an AI engine configured to identify classifiers that can be used to detect the identified objects based on the one or more rules generated by the rule formulator unit, where dominant and subordinate objects in the one or more rules from the rule formulator unit serve as identifiers for classifier selection by the AI ​​engine, select the identified classifiers from a bank of available object classifiers and activity classifiers, process data captured by the selected sensors using the selected classifiers, configure one or more expert system rules within the AI ​​engine of the process parallelization system according to the objects, operators, modifiers, and links identified in the received insight request, and process output from the selected classifiers using the expert system rules. Additionally, the core processor also includes an IDP formulator unit configured to compile insight data packets (IDPs) using outputs of the selected classifiers obtained from the AI ​​engine by processing with the expert system rules, and distribute the IDPs to multiple client systems.

[0004] In another aspect of the present disclosure, a method of process parallelization using a process parallelization system is provided. The method includes receiving an insight request from one of a plurality of client systems of the process parallelization system. The method further includes analyzing the received insight request to identify objects, operators, modifiers, and links therein. Furthermore, the method further includes identifying sensors and locations required to capture measurement variables corresponding to the identified objects. Furthermore, the method further includes selecting the identified sensors from a bank of available sensors and capturing data obtained by the selected sensors. Furthermore, the method further includes identifying classifiers that can be used to detect the identified objects and selecting the identified classifiers from a bank of available object classifiers and activity classifiers. Furthermore, the method further includes processing the data captured by the selected sensors using the selected classifiers. Furthermore, the method further includes configuring expert system rule(s) within an AI engine of the process parallelization system according to the identified objects, operators, modifiers, and links of the received insight request, and processing output from the selected classifiers using the expert system rules. Additionally, the method further includes compiling an IDP using the output of the selected classifier obtained from processing with the expert system rules, and distributing the IDP to a plurality of client systems of the process parallelization system.

[0005] In yet another aspect of the present disclosure, the aspects disclosed herein are also directed to a non-transitory computer-readable medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform the process parallelization methods disclosed herein.

[0006] It will be appreciated that features of the present disclosure are susceptible to being combined in various combinations without departing from the scope of the present disclosure as defined by the appended claims.

[0007] The above summary of the invention, as well as the following detailed description of illustrative embodiments, will be better understood when read in conjunction with the accompanying drawings. For purposes of illustrating the disclosure, typical constructions of the disclosure are shown in the drawings. However, the disclosure is not limited to the specific methods and utilities disclosed herein. Moreover, those skilled in the art will understand that the drawings are not to scale. Wherever possible, similar elements are designated by like reference numerals. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram of a process parallelization system according to one aspect of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of a core processor for the process parallelization system of FIG. 1 according to one embodiment of the present disclosure. [Figure 3] FIG. 3 is a schematic diagram illustrating a database of a rule formulator within a query processor of the core processor shown in FIG. 2 according to one embodiment of the present disclosure. [Figure 4] FIG. 3 is a schematic diagram of an AI engine in a core processor for the process parallelization system from FIG. 2 according to one embodiment of the present disclosure. [Figure 5] 1 is a flowchart of a method for process parallelization according to one aspect of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] In the accompanying drawings, underlined numbers are employed to represent the item in which the underlined number is located or the item to which the underlined number is adjacent. Non-underlined numbers refer to items identified by a line linking the non-underlined number and the item. When numbers are not underlined and are accompanied by an associated arrow, the non-underlined number is used to identify the general item to which the arrow points.

[0010] The following detailed description illustrates aspects and ways in which the present disclosure can be implemented. While best modes of carrying out the disclosure are disclosed, those skilled in the art will recognize that other ways for carrying out or practicing the disclosure are possible.

[0011] The present invention addresses the problem of the widening gap between the pipeline of available AI expertise and the demand for AI solutions by providing a mechanism for the reuse and reconfiguration of AI assets and access to AI solutions by multiple users of AI expertise.

[0012] 1, a process parallelization system 10 according to one embodiment of the present disclosure is depicted. The process parallelization system 10 includes a core processor 12 communicatively coupled to a plurality of sensors 14 a, 14 b, 14 c, 14 d, 14 e, and 14 f. The core processor 12 is further communicatively coupled to a data packet distribution system 16 and to a plurality of client systems 18 a, 18 b, and 18 c via corresponding two-way application programming interfaces (APIs) 20 a, 20 b, and 20 c.

[0013] Sensors 14a, 14b, 14c, 14d, 14e, and 14f include any sensing or monitoring device, including, but not limited to, physical sensors such as video cameras, temperature sensors, weight scales, pressure sensors, and humidity sensors; chemical sensors such as pH sensors and carbon dioxide sensors; and biological / biochemical sensors such as glucose sensors and pathogen sensors. For example, an observed scene including a checkout area may be monitored by one or more video cameras mounted at predefined positions relative to the checkout area such that the checkout area is located within the field of view of each video camera. Similarly, an observed scene including a standing-sized refrigerator may include a temperature sensor mounted at a predefined position to detect changes in the refrigerator temperature, and a video camera mounted at a predefined position at the opening / entrance to the refrigerator to detect movement of people and / or items.

[0014] Those skilled in the art will understand that the above-described sensors are provided for illustrative purposes only. In particular, those skilled in the art will appreciate that the presently disclosed process parallelization system 10 is not limited to the above-described sensors. Rather, the presently disclosed process parallelization system 10 may similarly and / or equivalently be used with other types of sensing devices capable of detecting changes in physical, chemical, and / or biological attributes of the observed system.

[0015] 2 in conjunction with FIG. 1, core processor 12 includes a query processor 22 communicatively coupled to an artificial intelligence (AI) engine 24, a sensor selector unit 26, and a record formulation unit 28. Query processor 22 includes a semantic analyzer 30, a rule formulator unit 32, and a sensor identifier unit 34.

[0016] Core processor 12 is configured to receive and handle request data packets (RDPs) from client systems 18a, 18b, and 18c via query processor 22. The RDPs include requests for user-specified insights about the user-specified observed scene. The insights relate to a high-level understanding of the observed scene derived from sensor data collected from the observed scene. The scope and parameters of the high-level understanding are established from criteria set in the received request, which may include, without limitation, prior knowledge of the application domain, habits and general or specific practices expected to occur or follow within the application domain or trends, and the type and availability of sensor data about the observed scene in the relevant application domain.

[0017] Take the retail environment as an example. A self-checkout store (SCO) non-scan event is one in which a customer appears to scan an item at a self-checkout but actually covers the item's barcode or otherwise moves the item around the self-checkout scanner, resulting in the item not being registered by the self-checkout. A SCO walk-through event is one in which a customer at a self-checkout fails to scan one or more, up to all, of the items in their basket or cart, and as a result exits the self-checkout with items in their basket or cart that they have not paid for.

[0018] A retailer may request a first insight via a first query regarding "the number of SCO non-scan events occurring at the self-checkout registers located closest to the store exits in geographic zone 1 between 4:00 PM and 9:00 PM local time." Additionally or optionally, the retailer may request a second insight regarding "the number of SCO non-scan events involving baby food products occurring at the self-checkout registers located closest to the store exits in geographic zone 1 between 4:00 PM and 9:00 PM local time." Additionally or optionally, the retailer may request a third insight regarding "the number of instances in which a person involved in a SCO walk-through event successfully exited a store in geographic zone 1 with an unpaid item between 4:00 PM and 9:00 PM local time."

[0019] Hereinafter, for ease of understanding, the explanation of how the process parallelization system 10 operates will be provided in conjunction with the exemplary insight requests described above. However, it should be noted that the exemplary insight requests described above are provided for illustrative purposes only. In particular, those skilled in the art will recognize that the process parallelization system 10 is not limited to responding solely to the exemplary insight requests disclosed herein. In fact, the process parallelization system 10 of the present disclosure can be used to respond to any type of insight request that may be formulated by a user for a given application domain.

[0020] The insight request may be formulated in RDP in natural language. Additionally or optionally, the insight request may be formulated in a defined format established according to known parsing syntax and grammar, including parsing expression grammar (PEG), link grammar, and top-down parsing language (TDPL). Those skilled in the art will recognize that the parsing languages ​​and grammars described above are merely exemplary and, therefore, non-limiting of the present disclosure. In particular, those skilled in the art will recognize that the process parallelization system 10 of the present disclosure is not limited to the parsing language and grammar. Rather, the process parallelization system 10 of the present disclosure can be used with any parsing language and grammar known in the art to be capable of expressing a user's insight request in a computer-understandable form.

[0021] Moreover, insight requests may be formulated by the user as a selection of objects, operators, and modifiers for the request from a bank of displayed objects, operators, and modifiers, thereby eliminating the need for the user to have prior programming knowledge or experience, including knowledge or experience of parsing languages, grammars, or formal logic representations.

[0022] The RDP also includes an identifier for the client system 18a, 18b, and 18c from which the request data packet originated, and an indicator of whether the insight generated in response to the query should be shared with the remaining client systems 18a, 18b, and 18c in the process parallelization system 10, or whether the insight should be accessible only to the particular client system 18a, 18b, and 18c from which the RDP originated. Taking the retail SCO store scenario described above as an example, a given retailer may not want competitors to know about SCO non-scan events or SCO walk-through events that occur within the store. In contrast, consider a query about a public space, such as "the number of people walking dogs and smelling red roses in Park A." In the current case, sharing the response to the insight request may be in the public interest, so the response may be shared with all client systems 18a, 18b, and 18c in the process parallelization system 10.

[0023] The query processor 22 is configured to receive insight requests from the received RDP via a semantic analyzer 30. The semantic analyzer 30 is configured to semantically analyze the received insight requests and identify dominant subject(s) and subordinate subject(s) and connections therebetween. The semantic analyzer 30 is further configured to identify location(s) associated with the dominant subject(s) and subordinate subject(s) from the received insight requests.

[0024] Let's take the first example of a requested insight: "The number of SCO non-scan events that occur at the self-checkout closest to the store exit in geographic zone 1 between 4:00 PM and 9:00 PM local time." The dominant target in this example is the SCO non-scan event. The location associated with the dominant target is the self-checkout closest to the store exit.

[0025] Similarly, consider a second example of a requested insight: "The number of SCO non-scan events involving baby food products that occur between 4:00 PM and 9:00 PM local time in geographic zone 1 at the self-checkout register located closest to the store exit." The dominant object of the requested insight is the SCO non-scan event, and the subordinate object is the baby food product. The location associated with the dominant object is the checkout register closest to the store exit. In the example just described, the location associated with the subordinate object is the same as the dominant object. However, one skilled in the art will understand that the location associated with the subordinate object may differ from the location associated with the dominant object, depending on the formulation of the received insight request.

[0026] Similarly, consider a third example of a requested insight: "The number of instances in which a person involved in a SCO walkthrough event successfully exits a store in Geographic Area 1 with unpaid items between the hours of 4:00 PM and 9:00 PM local time." In the example just given, the first dominant object is "SCO walkthrough event" and the second dominant object is "person leaving." The second dominant object is constrained by the geographic requirement that the leaving or exit be from a store exit. Because a SCO walkthrough event is performed by a person, in the example just given, the link between the first dominant object and the qualified second dominant object is the person involved in both the SCO walkthrough event and leaving the store exit. In other words, in this example, the same person must be involved in both the SCO walkthrough event and leaving the store exit. The locations associated with the first dominant object and the qualified second dominant object are the cash register closest to the store exit and the store exit, respectively. The detection of the same person at different locations is done by a re-identification algorithm, as will be described later.

[0027] The semantic analyzer 30 is configured to transmit the identified dominant object(s) (and / or modified dominant object(s)) and the identified subordinate object(s) (and / or modified subordinate object(s)), and the connections between them, to the sensor identifier unit 34. The semantic analyzer 30 is further configured to transmit the location(s) associated with the dominant object(s) and subordinate object(s) (and / or modified dominant object(s) and subordinate object(s)) to the sensor identifier unit 34.

[0028] The sensor identifier unit 34 is configured to receive the identified dominant object(s), subordinate object(s) (and / or the modified dominant object(s) and subordinate object(s)), and the connections therebetween, from the received insight request. The sensor identifier unit 34 is further configured to receive location(s) associated with the dominant object(s) and subordinate object(s) (and / or the modified dominant object(s) and subordinate object(s)). The sensor identifier unit 34 is further configured to establish measurement variables and corresponding measurement locations from the received dominant object(s), subordinate object(s) (and / or the modified dominant object(s) and subordinate object(s)), the connections therebetween, and the location(s) associated with the dominant object(s) and subordinate object(s) (and / or the modified dominant object(s) and subordinate object(s)).

[0029] Let's take a first example of a desired insight: "The number of SCO non-scan events occurring at the self-checkout register closest to the store exit in geographic zone 1 between 4:00 PM and 9:00 PM local time." Detecting SCO non-scan events may be done by comparing video footage capturing customer activity at the self-checkout register with the log of items registered by the self-checkout register. Thus, in the example just described, the measurement variables needed to detect the dominant target are the video footage captured by the video camera(s) and the transaction log from the self-checkout register. The measurement locations corresponding to the measurement variables are the self-checkout register closest to the store exit and a location close enough to the register so that the register is visible in the captured video footage.

[0030] Let's take a second example of a desired insight: "The number of SCO non-scan events involving baby food products that occur at the self-checkout registers located closest to the store exits in geographic zone 1 between 4:00 PM and 9:00 PM local time." In the example just described, the measurement variables needed to detect the dominant target are video footage captured by a video camera(s) and transaction logs from the self-checkout registers. The measurement locations corresponding to the measurement variables are the self-checkout registers closest to the store exits and locations close enough to the registers so that the registers are visible in the captured video footage. In the example just described, the measurement variables needed to detect the subordinate target are video footage captured by a video camera(s) and transaction logs from the self-checkout registers. The measurement locations corresponding to the measurement variables are the self-checkout registers closest to the store exits and locations close enough to the baby food products at the self-checkout registers so that the baby food products are visible in the captured video footage.

[0031] Similarly, consider a third example of a desired insight: "The number of instances in which a person involved in a SCO walkthrough event successfully exited a store in geographic area 1 with unpaid items between the hours of 4:00 PM and 9:00 PM local time." Detecting a SCO walkthrough event may be done by comparing video footage captured of a customer's cart or basket at a self-checkout counter with a log of items registered by the self-checkout counter. Similarly, detecting a person leaving a store exit may be done through analysis of captured video footage of the store exit. Thus, in the example just described, the measurement variables needed to detect the first dominant object and the qualifying second dominant object are the video footage captured by the video camera and the transaction log from the self-checkout counter. The measurement locations corresponding to the measurement variables are the self-checkout counter closest to the store exit, a first location close enough to the checkout counter so that the cart and basket next to the checkout counter are visible in the video footage captured therefrom, and a second location close enough to the store exit so that the exit and the cart passing by are visible in the video footage captured therefrom.

[0032] In the example just described, the link between the first dominant object and the second dominant object it qualifies is the person involved in both the SCO walkthrough event and leaving the store exit. The measurement variables needed to detect the just described link are video footage of the person at the self-checkout closest to the store exit and video footage of the person crossing the store exit. The measurement locations corresponding to the measurement variables are a location close enough to the cash register to allow capture of identifying features of the person at the cash register, and a location close enough to the store exit to allow capture of identifying features of the person crossing the store exit.

[0033] The sensor identifier unit 34 is configured to use the measurement variables and measurement locations to identify one or more of the sensors 14a-14f whose output is required to provide the requested insight. The sensor identifier unit 34 is further configured to send the identifier(s) (Sensor_ID) of the identified sensor(s) to the sensor selector unit 26. The operation of the sensor selector unit 26 will be described below.

[0034] The semantic analyzer 30 is further configured to identify temporal, geographic, and / or other attributes associated with the dominant subject(s) of the received insight request. Taking a first example of a requested insight, “The number of SCO non-scan events occurring at the self-checkout closest to the store exit in geographic zone 1 between 4 PM and 9 PM local time,” the temporal attribute associated with the identified dominant subject is the time period between 4 PM and 9 PM local time. Similarly, the geographic attribute and / or other additional attributes associated with the identified dominant subject is geographic zone 1. Similarly, taking a second example of a requested insight, “The number of instances in which a person involved in a SCO walk-through event successfully exited the store in geographic zone 1 with an unpaid item between 4 PM and 9 PM local time,” the temporal attribute associated with the identified first and second dominant subjects is the time period between 4 PM and 9 PM local time. Similarly, the geographic and / or other additional attributes associated with the identified first and second dominant subjects is geographic zone 1.

[0035] The semantic analyzer 30 is configured to send the identified dominant object(s), subordinate object(s) and links therebetween, as well as the identified temporal, geographical and / or other further attributes of the received insight request to the rule formulator unit 32.

[0036] 3, the rule formulator unit 32 includes one or more databases of numerical and statistical operators 54 operable on measurements corresponding to the dominant subject(s) and subordinate subject(s) of the received insight request. The rule formulator unit 32 further includes at least one database of logical operators 54 operable on measurements corresponding to the dominant subject(s), subordinate subject(s) 52, and the link(s) 58 therebetween.

[0037] The rule formulator unit 32 is configured to analyze the received insight request and detect the presence of indicator(s) of the required result of the received insight request, as well as connections between dominant subject(s), subordinate subject(s), and the required result. The rule formulator unit 32 is adapted to select, from its database 54, an operator corresponding to or associated with the indicator(s) of the required result. The rule formulator unit 32 is adapted to analyze the received insight request and detect the presence of modifier(s) 56 of the dominant subject(s) 52, subordinate subject(s) 52, and the link 58 therebetween. The rule formulator unit 32 is adapted to select, from the database 54, an operator corresponding to the detected modifier 56 according to the expression of the modifier in the received insight request.

[0038] The rule formulator unit 32 is further configured to assemble the retrieved operators 54, the retrieved modifiers 56, the received dominant and subordinate objects 52 and the links 58 therebetween, as well as the temporal, geographical and / or other further attributes of the received insight request into rules 50 that conform to the semantic logic of the received insight request provided by the semantic analyzer 30. Specifically, the rules may be formulated, for example, as a linear expression or a decision tree structure, and the result output from the rules may be numerical, graphical, textual or Boolean.

[0039] Let's take the first example of a requested insight, namely, "The number of SCO non-scan events that occur at the self-checkout closest to the store exit, in geographic zone 1, between 4:00 PM and 9:00 PM local time." In the example just given, the desired outcome indicator is the "number" of dominant objects (i.e., SCO non-scan events). Hence, the operator corresponding to the just given indicator is the summation operator. The dominant object modifier in the just given example is a restriction associated with further geographic and temporal attributes of the requested insight. Hence, the operator corresponding to the modifier in this example is a logical "AND" function. Thus, the rule for the just given example might be expressed as follows:

[0040]

number

[0041] In the rule just stated, y is the output, x is the measurement, NSE is the detected non-scan event, Geo() is the geographic dependency, g1 is geographic zone 1, BT() is the time dependency, which indicates the period between two specified times, t1 is 4 PM local time and t2 is 9 PM local time.

[0042] Let's take a second example of a requested insight: "The number of SCO non-scan events involving baby food products that occur at the self-checkouts located closest to the store exits in geographic zone 1 between 4:00 PM and 9:00 PM local time."

[0043] In the example just given, the operator is the same as in the first example. However, the modifier includes an additional element, namely the presence of a detected subordinate object. Thus, the rule for the example just given might be expressed as follows:

[0044]

number

[0045] In the just stated rule, the parameters y, x, NSE, Geo(), g1, BT(), t1 are the same as in equation (1), and Obj is the detection of inferior objects (i.e., baby food products).

[0046] Let's take a third example of a requested insight: "The number of instances in which a person included in a SCO walkthrough event successfully exited a store in geographic area 1 with unpaid items between 4:00 PM and 9:00 PM local time." In the example just given, the required outcome indicator is the "number" of a first dominant object (i.e., SCO walkthrough events) linked with the number of a second dominant object (i.e., exiting people) qualified by the departure being from a store exit. Thus, the operator corresponding to the just given indicator is the summation operator. The link between the first dominant object and the second dominant object is the reidentification of the same person included in both the SCO walkthrough event and the departure from the store exit. In the just given example, the qualifier for the second dominant object is the geographic requirement that the departure be from a store exit. In the just given example, the qualifiers for the linked first dominant object and the qualified second dominant object are constraints associated with further geographic and temporal attributes of the requested insight. Therefore, the operator corresponding to the modifier in this example is the logical "AND" function. Thus, the rule in the example just given might be expressed as:

[0047]

number

[0048] In the just stated rule, y is the output, x1 and x2 are measurements associated with the first dominant object and the second dominant object, respectively. SWT is the detected SCO walkthrough event, Geo() is the geographic dependency, Exit is the store exit, and g1 is geographic zone 1. ReID is the re-identification of the detected person. BT() is the time dependency indicating the period between two specified times, where t1 is 4 PM local time and t2 is 9 PM local time.

[0049] Those skilled in the art will appreciate that the above examples of received user insight requests and corresponding rule formulations are provided for illustrative purposes only. In particular, those skilled in the art will appreciate that the process parallelization system 10 of the present disclosure is not limited to the examples. Rather, the process parallelization system 10 of the present disclosure can be used to implement any formulation suitable for expressing a received insight request to enable the selection and processing of measurements of variables from an observed scene to enable the generation of an appropriate response in response to the insight request.

[0050] Returning to Figure 2, the rule formulator unit 32 is configured to send the generated rule(s) to the AI ​​engine 24. As will be described later, the dominant and subordinate objects in the rules from the rule formulator unit 32 are identifiers used for selection of a classifier (not shown) in the AI ​​engine 24, and the rules themselves form part of the body expert system rules (not shown) of the AI ​​engine 24. The semantic analyzer 30 is further configured to detect and identify contextual information, if present, in the received insight request. The semantic analyzer 30 is configured to send the identified contextual information to the AI ​​engine 24 for use in interpreting output from the classifier, as will be described later.

[0051] The sensor selector 26 is configured to select and combine outputs from one or more sensors 14a-14f according to the identifiers received from the sensor identifier unit 34. Specifically, referring to FIG. 2 in conjunction with FIG. 1, outputs from sensors 14a, 14b, 14c, 14d, 14e, and 14f may be clustered into different configurable combinations to form corresponding sensor families F1, F2, and F3. In this example, outputs from sensors 14a, 14b, and 14c are combined into sensor family F1. Similarly, outputs from sensors 14c and 14d are combined into sensor family F2. Furthermore, outputs from sensors 14e and 14f are combined into sensor family F3. Output from a single sensor may be included in one or more sensor families. For example, output from sensor 14c is included in sensor family F1 and sensor family F2. In contrast, output from sensor 14a is included only in sensor family F1.

[0052] The outputs from the sensors 14 may be combined into sensor families according to sensor type. For example, a first sensor family may include outputs from multiple temperature sensors, and a second sensor family may include outputs from multiple pressure sensors. Similarly, the outputs from the sensors may be combined into sensor families according to application domain. For example, a first sensor family may include outputs from multiple video cameras mounted in a cash register area, and a second sensor family may include outputs from multiple video cameras mounted in a kitchen area.

[0053] Alternatively, outputs from sensors may be combined into sensor families according to the category of occurrence to be detected. For example, a first sensor family may include outputs from multiple video cameras positioned to detect the occurrence of an event where an item is not scanned at a cash register, and a second sensor family may include outputs from multiple video cameras positioned to detect food items being dropped onto a kitchen floor. Further alternatively, outputs from sensors may be combined into sensor families according to a combination of multiple criteria, i.e., multiple combination criteria, such as the location of each sensor and its corresponding sensor type. For example, a sensor family may include outputs from all video cameras mounted in the cash register area closest to the entrance / exit from a retail store within a specified geographic area.

[0054] Those skilled in the art will understand that the number and arrangement of sensor families described above are provided for illustrative purposes only. In particular, those skilled in the art will appreciate that the process parallelization system 10 of the present disclosure is in no way limited to the above-described number and arrangement of sensor families. Rather, the process parallelization system 10 of the present disclosure can use any number, arrangement, and configuration of sensor families required by an end user or client associated with client systems 18 a, 18 b, and / or 18 c to monitor an observed scene.

[0055] Continuing from the above, the number and composition of sensor families F1, F2, F3 created by sensor selector 26 is determined by the measurement variables and measurement locations determined from the received insight request by sensor identifier unit 34 of core processor 12. In effect, sensor selector 26 creates a mask over the outputs from sensors 14a-14f, selecting and combining them to meet the requirements of the received insight request. Thus, the mask is essentially a representation of the measurement variables of the rules derived by rule formulator unit 32 from the received insight request.

[0056] Taking the first example of a requested insight, namely, "the number of SCO non-scan events that occur at the self-checkout registers closest to the store exit in geographic zone 1 between 4:00 PM and 9:00 PM local time," the sensor(s) selected by sensor selector 26 include video camera(s) located sufficiently close to the cash registers closest to the store exit so that the cash registers are visible in video footage captured by the video camera(s). The sensor(s) selected by sensor selector 26 further include transaction logs for the corresponding cash registers.

[0057] Similarly, taking the second example of a requested insight, namely, "The number of SCO non-scan events involving baby food products that occur in geographic zone 1 at the self-checkout register located closest to the store exit between 4:00 PM and 9:00 PM local time," the sensor(s) selected by sensor selector 26 include video camera(s) located sufficiently close to the cash register(s) closest to the store exit so that the cash register and the baby food products at the cash register are visible in video footage captured by the video camera(s). The sensor(s) selected by sensor selector 26 further include the transaction log for the corresponding cash register.

[0058] Further, taking the third example of a requested insight, namely, "The number of instances in which a person included in a SCO walkthrough event successfully exited a store in geographic area 1 with an unpaid item between the hours of 4:00 PM and 9:00 PM local time," the sensors selected by sensor selector 26 include: ● video camera(s) located sufficiently close to the cash register(s) closest to the store exit so that the cash register(s) and the cart(s) and / or basket(s) next to the cash register(s) are visible in the video footage captured by the video camera(s); ● video camera(s) located sufficiently close to the store exit so that the exit and the cart passing through it are visible in the video footage captured by the video camera(s); and ● The transaction log of the corresponding cash register.

[0059] If a sensor is not available that generates a measurement variable that meets the requirements of a received insight request, a dialogue data packet (DDP) is delivered to the client system 18a, 18b, or 18c that originated the insight request, as described below.

[0060] The stream of data captured from each sensor family F1, F2, F3 is sent to the AI ​​engine 24. In other words, the AI ​​engine 24 is configured to receive the outputs from the sensor families created by the sensor selector 26.

[0061] Referring to FIG. 4, the AI ​​engine 24 includes an object detection block 40 and an activity detection block 42 communicatively coupled to an expert system block 44, and the object detection block 40 includes a plurality of object recognition engines ODet1, ODet2, . . . , ODet n and the activity detection block 42 includes a plurality of activity recognition engines ADet1, ADet2, . . . , ADet m Includes.

[0062] Object recognition engines ODet1, ODet2, ..., ODet nは, and corresponding multiple object recognition algorithms (not shown). In one aspect, the object recognition algorithms include a deep neural network whose architecture is substantially based on YOLOv4 (as described in A. Bochkovskiy, C.Y. Wang and H.Y. M. Liao, 2020 arXiv:2004.10934) or EfficientDet (as described in M. Tan, R. Pang and Q.V. Le, EfficientDet: Scalable and Efficient Object Detection, 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 10778-10787). However, those skilled in the art will understand that the object recognition deep neural network architecture is provided for illustrative purposes only. In particular, those skilled in the art will understand that the process parallelization system 10 of the present disclosure is not limited to deep neural network architectures. Rather, the process parallelization system 10 of the present disclosure can be used with any object recognition architecture and / or training algorithm suitable for detecting / recognizing and classifying specific objects in images or video frames.

[0063] The objects may include people, vehicles, pallets, food, or other man-made objects as designated by an operator. Those skilled in the art will appreciate that the above-described objects are merely illustrative in nature and, therefore, are provided herein for illustrative purposes only. In particular, those skilled in the art will appreciate that the process parallelization system 10 of the present disclosure is not limited to detecting objects alone. Rather, the process parallelization system 10 of the present disclosure can be used to detect and recognize the presence of any type of specific object within an image or video frame.

[0064] Activity recognition engines ADet1, ADet2,..., ADet mThe system includes multiple corresponding activity recognition algorithms (not shown). For example, the activity recognition algorithms may detect pose changes of a person's articulated body detected in a captured video stream, as described in U.S. Patent No. 10,937,185. Another suitable mechanism for pose detection is to use the UniPose neural network architecture, as disclosed in Artacho, B. and Savakis, A., 2020. UniPose: Unified Human Pose Estimation in Single Images and Videos. arXiv preprint arXiv:2001.08095. The described neural network uses past information to account for temporal factors when estimating joint movements. The described network also accounts for blur and occlusion. The detected pose changes are then correlated with specified actions, such as bending, stretching, running, and other commonly known postures and poses commonly performed by humans, as known in the art.

[0065] Those skilled in the art will understand that the above-described pose detection algorithm is provided for illustrative purposes only. In particular, those skilled in the art will understand that the process parallelization system 10 of the present disclosure is not limited to pose detection algorithms. Rather, the process parallelization system 10 of the present disclosure can be used with any pose detection architecture and / or training algorithm suitable for detecting and classifying objects, particularly living objects such as humans and / or animals, detected in captured video footage of an observed scene. Those skilled in the art will further understand that the above-described activities determined from detected pose changes are also provided for illustrative purposes only. In particular, those skilled in the art will understand that the process parallelization system 10 of the present disclosure is not limited to activity detection. Rather, the process parallelization system 10 of the present disclosure can be used to detect any specified activity that may be correlated with detected pose changes of humans and / or animals detected in captured video footage of an observed scene.

[0066] Object recognition engines ODet1, ODet2, ..., ODet n and activity recognition engines ADet1, ADet2, ···, ADet m The object recognition algorithm (not shown) and the activity recognition algorithm (not shown) are respectively stored in the corresponding training data databases DB1 to DB2. p It may be trained on data housed in a matrix, where p ≤ m + n. For example, training a Unipose network may be implemented with a single configuration, as outlined in Mykhaylo Andriluka, Leonid Pishchulin, Peter Gehler, and Bernt Schiele. 2D human pose estimation: New benchmark and state of the art analysis, In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014.

[0067] Training data database DB1~DB pは , which contain past examples of the objects and / or activities to be detected. The examples are obtained from video footage captured from a variety of settings and observation subjects. For example, data used to detect bending and stretching activities may include examples of people of different genders, ages, skin colors, and overall body types. In any case, the training data databases DB1 through DB2 p The assembly of examples in improves heterogeneity and trains the object recognition engines ODet1, ODet2, . . ., ODet n and activity recognition engines ADet1, ADet2, ···ADet m To further improve generalization capabilities, the object recognition engines ODet1, ODet2, . . ., ODet n , and activity recognition engines ADet1, ADet2, ···, ADet m, training data database DB1 to DB p The examples contained in the training data databases DB1 through DB2 are also captured under a variety of environmental conditions, including lighting, wind, and other conditions known to those skilled in the art that may change over time. p To maintain the currency of the table, further examples may be added thereto from data captured by sensors 14a, 14b, 14c, 14d, 14e and 14f, as described below.

[0068] The AI ​​engine 24 may further include a plurality of contextualizer units C1-Cq. The contextualizer units C1-Cq may include a plurality of user-provided contextual information regarding implicitly or explicitly articulated rules or habits followed by participants in the observed scene. For example, the contextualizer units C1-Cq may include contextual information identified in an insight request received by the semantic analyzer 30 and sent to the AI ​​engine 24. The contextual information may be provided by the object recognition engines ODet1, ODet2, ..., ODet n and activity recognition engines ADet1, ADet2, ···, ADet m , and ADet1, ..., ADet2, .... In one aspect, the contextual information may be used to exclude detected activities from further consideration or reporting to the user unless otherwise specified. For example, leaving for a coffee break at a given time may be common in the observed scene for everyone working there. Thus, the activity recognition engines ADet1, ADet2, ..., ADet m In detecting the movement of a person in an observed scene towards an exit at a specified time, it is possible to infer that the cause of the detected movement is a habitual coffee break whose occurrence may not be of interest to the user.

[0069] In other aspects, the contextual information may be used to highlight specific detected activity for further analysis in more detail or reporting to the user. Using the previous example, the detection of a first person re-entering an observed scene where no other person is working there, and the first person's handling of personal belongings of other people, may be factors of concern requiring further analysis.

[0070] Those skilled in the art will appreciate that the above-described scenarios and associated contextual interpretations and results are for illustrative purposes only. In particular, those skilled in the art will appreciate that the process parallelization system 10 of the present disclosure is not limited to these scenarios and associated contextual interpretations and results. Rather, the process parallelization system 10 of the present disclosure may include object recognition engines ODet1, ODet2, . . . , ODet n and activity recognition engines ADet1, ADet2, ···, ADet m It can be used with any type of scenario and associated contextual interpretation and results as provided by the user to facilitate interpretation of activities and / or objects detected by the system.

[0071] The AI ​​engine 24 further includes an expert system block 44 including a plurality of expert systems ES1 to ESs. The expert systems ES1 to ESs include a plurality of object recognition engines ODet1, ODet2, . . . , ODet n and multiple activity recognition engines ADet1, ADet2,···, ADet m The expert systems ES1 to ESs include one or more rules received from the rule formulator unit 32. Specifically, the expert systems from the expert system block 44 are communicatively coupled to one or more object recognition engines ODet1, ODet2, ..., ODet selected from the object detection block 40 and / or the activity detection block 42. n and / or activity recognition engines ADet1, ADet2, ..., ADet mThe object recognition engine(s) and activity recognition engine(s) selected correspond to the dominant and / or subordinate objects specified in the rules of the expert system 44.

[0072] Therefore, for the first example of a requested insight, i.e., “The number of SCO non-scan events that occur at the self-checkout closest to the store exit in geographic zone 1 between 4 PM and 9 PM local time,” the corresponding expert system will be composed of activity recognition engines ADet1, ADet2, …, ADet m The output from the activity recognition engine is generated in response to: (a) Video footage captured and received by a video camera(s) located in sufficient proximity to the cash register nearest the store exit so that the cash register is visible in the video footage; and (b) Transaction logs received from cash registers.

[0073] Similarly, let us take the second example of a desired insight, namely, "The number of SCO non-scan events involving baby food products that occur at the self-checkouts located closest to the store exits in geographic zone 1 between 4 PM and 9 PM local time." The corresponding expert system consists of activity recognition engines ADet1, ADet2, ... ADet m The output from the activity recognition engine is generated in response to: (a) received video footage captured by a video camera located in sufficient proximity to the cash register nearest the store exit that the cash register and the baby food products therein are visible in the video footage; and (b) Transaction logs received from cash registers.

[0074] Similarly, let us take the third example of a desired insight, namely, "the number of instances in which a person included in a SCO walkthrough event successfully exited a store in geographic area 1 with an unpaid item between the hours of 4 PM and 9 PM local time." The corresponding expert system consists of two activity recognition engines ADet1, ADet2, ..., ADet m An output from a first activity recognition engine trained to detect a walkthrough event of the SCO and a person leaving the store, respectively, is then generated in response. (a) received video footage captured by a video camera(s) located sufficiently close to the cash register nearest the store exit so that the cash register and any carts and / or baskets next to the cash register are visible in the video footage; and (b) Transaction logs received from cash registers.

[0075] The output from the second activity recognition engine is generated in response to received video footage captured by video camera(s) located sufficiently close to the store exit so that the exit and the cart passing through it are visible in the video footage.

[0076] Thus, the expert systems ES1 to ESs may be communicatively coupled to the or each of the contextualizer units C1 to Cq. The expert systems ES1 to ESs may further include dependency relationships associated with one or more items of contextual data received from the contextualizer units C1 to Cq.

[0077] The expert system rules are generated by the individual object recognition engines ODet1, ODet2, ..., ODet n and activity recognition engines ADet1, ADet2, ···, ADet mThe re-identification may include a link between the output of the SCO and the output of the store exit, which may be performed by an additional classifier. In the third example above, the link is the re-identification that the same person is involved in both the SCO walkthrough and leaving the store via the store exit. The re-identification operation is performed by a re-identification engine (not shown), which may employ a coarse-to-fine pyramid model in a lossy dynamic training approach implemented by a standard ResNet architecture or a BN-Inception architecture. Alternatively, the re-identification engine (not shown) may employ a SORT algorithm (such as that described in Bewley A, Ge Z., Ott L., Ramos F. and Upcroft B., Simple Online and Realtime Tracking, 2016 IEEE International Conference on Image ProcESsing (ICIP), Phoenix, AZ, 2016, pp. 3464-3468) or a DeepSort algorithm (such as that described in Porrello A, Bergamini L. and Calderara S., Robust Re-identification by Multiple View Knowledge Distillation, Computer Vision, ECCV 2020, Springer International Publishing, European Conference on Computer Vision, Glasgow, August 2020).

[0078] Those skilled in the art will appreciate that the example person re-identification algorithms described above are provided for illustrative purposes only. In particular, those skilled in the art will appreciate that the process parallelization system 10 of the present disclosure is in no way limited to the use of the person re-identification algorithms described above. Rather, the process parallelization system 10 of the present disclosure may be implemented using the individual object recognition engines ODet1, ODet2, . . . , ODet nand the output of the individual activity recognition engines ADet1, ADet2, ···, ADet m It can be used by any re-identification algorithm capable of supporting linking between the same person or things involved in moving between different activities or different geographic locations specified by the received insight request.

[0079] Furthermore, those skilled in the art will recognize that the individual object recognition engines ODet1, ODet2, . . . , ODet nの Output and activity recognition engines ADet1, ADet2, ···, ADet m It will also be appreciated that re-recognition is not the only way in which a given expert system rule may be executed. In particular, those skilled in the art will appreciate that the above discussion regarding re-recognition is provided to complete the description of the third example above. Thus, the above discussion regarding re-identification is provided for illustrative purposes only. In particular, those skilled in the art will appreciate that the process parallelization system 10 of the present disclosure may be implemented by linking the individual object recognition engines ODet1, ODet2, ..., ODet n and the individual activity recognition engines ADet1, ADet2, ···, ADet m It will be appreciated that the present disclosure is in no way limited to the use of re-identification to link the outputs of the individual object recognition engines ODet1, ODet2, ..., ODet n and activity recognition engines ADet1, ADet2, ···, ADet m It can be used by any algorithm that may support temporal, geographic or contextual linking of the output of the In the manner just described, AI engine 12 can be used to process requests received from multiple client systems 18a, 18b, and 18c in parallel through: ● simultaneous processing of measurements captured by different combinations and permutations of measurement sensors by multiple object recognition engines and / or activity recognition engines; ● Simultaneous use of outputs from different combinations and permutations of object recognition engines and / or activity recognition engines by multiple expert system rules configured according to each received request.

[0080] Object recognition engines ODet1, ODet2, ..., ODet n or activity recognition engines ADet1, ADet2, ···, ADet m If none of the above can detect the dominant or subordinate object of the received insight request, then in object detection block 40 or activity detection block 42, a dialogue data packet (DDP) is delivered to the client system 18a, 18b, or 18c that originated the insight request, as described below.

[0081] Combining Figure 4 with Figure 2, the output from the expert systems block 44 of the AI ​​engine 24 is sent to an insight data packet (IDP) formulator 38. The IDP formulator 38 is configured to establish a link between the output received from the individual expert systems in the AI ​​engine 24 and the RDP from which the insight requests expressed by the rules of the expert systems originate.

[0082] Specifically, IDP formulator 38 is configured to compile an IDP from: (a) The output from the corresponding expert system; (b) the received outputs of the individual expert systems in the AI ​​engine 24; and (c) A tag that matches the client system identifier and privacy status indicator of the corresponding request data packet RDP.

[0083] Specifically, the IDP includes a client system identifier tag that identifies the client system 18a, 18b, or 18c from which or to which the request data packet corresponding to the IDP originated. The IDP also includes a Boolean privacy tag, where a value of 0 indicates that the expert system output may be shared with the rest of the client systems 18a, 18b, and 18c of the process parallelization system 10, and a value of 1 indicates that the expert system output is accessible only by the client system 18a, 18b, or 18c from which or to which the request data packet originated.

[0084] To later support auditing and modification of insight requests received from individual client systems 18a, 18b, and / or 18c and rules formulated from those insight requests, core processor 12 includes a database 36 configured to house details of received RDPs.

[0085] Combining Figures 2 and 1, the IDPs created by IDP formulator 38 are distributed over packet distribution system 16. Issuance may be preceded by a broadcast distribution packet distribution system 16 providing advance warning to client systems 18a, 18b, and 18c of the impending distribution of the IDP related to the particular received RDP.

[0086] Once distributed on packet distribution system 16, the IDP is distributed to all client systems 18a, 18b, and 18c (hereinafter simply referred to as "clients," "subscribing clients," or "subscribing client systems," and designated with the same reference numerals 18a, 18b, and 18c) that are subscribed to process parallelization system 10. Two-way APIs 20a, 20b, and 20c authorize subscribed client systems 18a, 18b, and 18c to access the IDP depending on the values ​​of the client system's privacy tag and identifier tag.

[0087] Specifically, if the privacy tag of the IDP evaluates to, for example, 0, all client systems 18a, 18b, and 18c may access the IDP through APIs 20a, 20b, and 20c to obtain output from the expert system included in the insight data packet (IDP). If the privacy tag of the IDP evaluates to, for example, 1, the client identifier tag of the insight data packet (IDP) is inspected by APIs 20a, 20b, and 20c of subscribing client systems 18a, 18b, and 18c. In the event of a match between the client identifier tag and the identifier of a subscribed client system, the subscribed client system may access the insight data packet (IDP) through the API to obtain output from the expert system included in the IDP.

[0088] Once circulated among client systems 18a, 18b, and 18c, the insight data packet (IDP) is returned to core processor 12, which may optionally examine the IDP's timestamp. If the time elapsed since the timestamp is less than a preconfigured threshold, core processor 12 may optionally redistribute the IDP in packet delivery system 16. Otherwise, core processor 12 may optionally store the IDP in database 36.

[0089] The core processor 12 may also create a DDP if sensors that generate measurement variables that meet the requirements of a received insight request are unavailable, or if the dominant or subordinate targets of the received insight request cannot be detected by any of the object recognition engines or activity recognition engines of the AI ​​engine 12. The DDP includes a privacy tag and a client identifier tag that can be established and used as described above in connection with the IDP. The DDP further includes a narrative explaining the lack of suitable sensor(s) and / or object recognition engine(s) or activity recognition engine(s) and requests the client systems 18a, 18b, and 18c to advise whether they wish to continue with the corresponding RDP, which may involve additional costs for establishing the required sensor(s) and / or object recognition engine(s) or activity recognition engine(s). Client systems that are granted access to the DDP according to the privacy tag and client identifier tag may include a response in the DDP along with the identifier of the responding client system. The resulting DDP may be distributed by the responding client system over a packet distribution system 16 where the DDP is returned to the core processor 12 for review by an operator of the process parallelization system 10 .

[0090] Referring to FIG. 5, a method 100 of process parallelization includes the following steps. receiving an insight request from one of a plurality of subscribed clients of the process parallelization system; - Analyzing incoming insight requests to identify subjects, operators, modifiers, and links 104; - Identifying sensors and locations required to capture measurement variables corresponding to the identified objects 106; - selecting the identified sensor from a bank of available sensors (not shown); - capturing data obtained by selected sensors 108; - identifying a classifier that can be used to detect the identified object 110; - selecting the identified classifier from a bank of available object and activity classifiers (not shown); - processing the data captured by the selected sensors with the selected classifiers 112; - constructing 114 a rule(s) of the expert system according to the objects, operators, modifiers, and links identified in the received insight request; - processing the output from the selected classifiers by expert system rules 116; - Compiling insight data packets from the results of processing by the expert system rules (not shown); and - Distributing the resulting insight data packets to subscribed clients of the process parallelization system 118.

[0091] Changes to the aspects of the present disclosure described above are possible without departing from the scope of the present disclosure as defined by the appended claims. Words such as "comprises," "including," "incorporating," "including," "having," "being," and the like, used to describe and claim the present disclosure, are intended to be construed in a non-exclusive manner, i.e., allowing for the presence of items, components, or elements not expressly recited. Furthermore, references to the singular are also construed as references to the plural.

Claims

1. A process parallelization system, comprising: a core processor communicatively coupled to a plurality of sensors; and a data packet distribution system coupled to a plurality of client systems through corresponding two-way application programming interfaces (APIs), the core processor comprising: a query processor configured to receive an insight request from one of the plurality of client systems, the query processor comprising: a semantic analyzer configured to analyze the received insight request to identify subjects, operators, modifiers, and links; and a sensor identifier unit configured to identify sensors and corresponding locations required to capture measurement variables corresponding to said identified objects; a rule formulator unit configured to generate one or more rules based on the received insight request; a query processor comprising: a sensor selector unit configured to select the identified sensor from the plurality of sensors for capturing data directly related to the identified object; Identifying classifiers that can be used to detect the identified objects based on the one or more rules generated by the rule formulator unit, wherein dominant and subordinate objects in the one or more rules from the rule formulator unit serve as identifiers for selection of classifiers by an artificial intelligence (AI) engine; selecting the identified classifier from a bank of available object and activity classifiers; Using the selected classifier, process the data captured by the selected sensor. configuring the one or more rules in an expert system of the AI ​​engine of the process parallelization system according to the objects, operators, modifiers, and links identified in the received insight request; Processing the output from the selected classifier using rules of the expert system. The AI ​​engine configured as above; compile an IDP using the outputs of the selected classifiers obtained from the AI ​​engine by processing them with rules of the expert system; Distributing the IDP to the plurality of client systems An Insight Data Packet (IDP) formulator unit configured as follows: A process parallelization system comprising:

2. 2. The process parallelization system of claim 1, wherein the semantic analyzer is configured to identify dominant subjects, subordinate subjects, any links therebetween, and temporal, geographic, and other additional attributes of the received insight request and send them to the rule formulator unit.

3. The rule formulator unit a database of numerical and statistical operators that can be used for measurements corresponding to the dominant and subordinate objects of the received insight request; a database of logical operators that can be used on measurements corresponding to the dominant object, the subordinate object, and the link between them; 3. The process parallelization system according to claim 2, further comprising:

4. 2. The process parallelization system of claim 1, wherein the number and composition of sensor families created by the sensor selector unit are determined by measurement variables and measurement locations determined from insight requests received by a sensor identifier unit of the core processor.

5. 2. The process parallelization system of claim 1, wherein if sensors are not available to generate measurement variables that meet the requirements of a received insight request, the core processor creates a dialog data packet (DDP) that is delivered to the client system from which the insight request originated.

6. 2. The process parallelization system of claim 1, wherein the AI ​​engine includes an object detection block and an activity detection block communicatively coupled to an expert system block having the expert system, the object detection block including a plurality of object recognition engines, and the activity detection block including a plurality of activity recognition engines.

7. 7. The process parallelization system according to claim 6, wherein the object recognition engine includes a corresponding plurality of object recognition algorithms, and the activity recognition engine includes a corresponding plurality of activity recognition algorithms.

8. 8. The process parallelization system of claim 7, wherein the AI ​​engine further comprises a plurality of contextualizer units containing a plurality of user-supplied contextual data relating to at least one of implicit and explicitly articulated rules and habits followed by subjects within an observed scene that are used to aid in the interpretation of objects and activities detected by the object recognition engine and activity recognition engine, respectively.

9. 9. The process parallelization system of claim 8, wherein the AI ​​engine includes an expert system block having a plurality of expert systems communicatively coupled to the plurality of object recognition engines and the plurality of activity recognition engines, the plurality of expert systems including one or more rules received from the rule formulator unit, and processes outputs from one or more of the plurality of object recognition engines and the plurality of activity recognition engines selected from the object detection block and the activity detection block such that the selected one or more object recognition engines and the selected one or more activity recognition engines correspond to dominant and subordinate objects specified in the one or more rules of the expert system.

10. 10. The process parallelization system of claim 9, wherein the rules of the expert system include linking the outputs of the individual object recognition engines and activity recognition engines included within the expert system of the AI ​​engine, the linking being performed by the expert system using additional classifiers.

11. The AI ​​engine is simultaneous processing by said plurality of object recognition engines and activity recognition engines of measurements captured by different combinations and permutations of measurement sensors; and a plurality of expert system rules configured according to each received request to simultaneously use the outputs from different combinations and permutations of the object recognition engines and activity recognition engines.

11. The process parallelization system of claim 10, wherein the process parallelization system can be used to process in parallel insight requests received from the plurality of client systems through at least one of:

12. 12. The process parallelization system of claim 11, wherein the IDP formulator unit is configured to establish a link between the received output from an individual expert system in the AI ​​engine and a request data packet (RDP) from which the insight request originated, as represented by the rule of the expert system.

13. The IDP formulator unit comprises: the output received from the expert system in the AI ​​engine; one or more tags that match the corresponding RDP client identifier and privacy status indicator; 13. The process parallelization system according to claim 12, wherein the system is configured to compile the IDP using

14. 14. The process parallelization system of claim 13, wherein the core processor includes a database configured to store received RDP details to facilitate at least one of auditing and remediation of the insight requests received from individual client systems and formulating rules from corresponding insight requests.

15. 15. The process parallelization system of claim 14, wherein the two-way API is configured to grant access to the IDP to one or more client systems from the plurality of client systems depending on values ​​of a privacy tag and an identifier tag for each client system.

16. 1. A method of process parallelization using a process parallelization system, comprising: receiving an insight request from one of a plurality of client systems of the process parallelization system; analyzing the received insight request to identify objects, operators, modifiers, and links; Identifying sensors and locations necessary to capture measurement variables corresponding to the identified objects; selecting the identified sensor from a bank of available sensors; capturing data obtained by the selected sensors; and identifying a classifier that can be used to detect the identified object; selecting the identified classifier from a bank of available object and activity classifiers; processing the data captured by the selected sensors with the selected classifiers; constructing, within an AI engine of the process parallelization system, rules for an expert system according to the objects, operators, modifiers, and links identified in the received insight request; processing the output from the selected classifier using rules of the expert system; compiling an insight data packet (IDP) using the outputs of the selected classifiers obtained from processing with rules of the expert system; distributing the IDP to the plurality of client systems of the process parallelization system; A method comprising:

17. analyzing the received insight request to identify objects, operators, modifiers, and links therein includes identifying dominant objects, subordinate objects, any links therebetween, and temporal, geographic, and other additional attributes of the received insight request; Identifying sensors and locations required to capture measurement variables includes determining the number and composition of sensor families created by the measurement variables corresponding to measurement locations determined from the received insight request, and if sensors are not available to generate measurement variables that meet the requirements of the received insight request, creating a dialog data packet (DDP) to be delivered to the client system from which the insight request originated.

17. The method of claim 16.

18. The AI ​​engine is an object detection block and an activity detection block communicatively coupled to an expert system block having the expert system, the object detection block including a plurality of object recognition engines, and the activity detection block including a plurality of activity recognition engines; a plurality of contextualizer units including a plurality of user-supplied context data relating to at least one of implicit and explicitly articulated rules and habits followed by subjects in a observed scene, the contextualizer units being used to assist in the interpretation of objects and activities detected by the object recognition engine and the activity recognition engine, respectively; 17. The method of claim 16, comprising:

19. The AI ​​engine is simultaneous processing by said plurality of object recognition engines and activity recognition engines of measurements captured by different combinations and permutations of measurement sensors; and a plurality of expert system rules configured according to each received request to simultaneously use the outputs from different combinations and permutations of the object recognition engines and activity recognition engines.

20. The method of claim 18, wherein the method can be used to process insight requests received from the plurality of client systems in parallel through at least one of:

20. When executed by the processors of a process parallelization system, receiving an insight request from one of a plurality of client systems of the process parallelization system; analyzing the received insight request to identify subjects, operators, modifiers, and links; Identifying sensors and locations required to capture measurement variables corresponding to the identified objects; selecting the identified sensor from a bank of available sensors; capturing data obtained by the selected sensors; identifying a classifier that can be used to detect the identified object; selecting the identified classifier from a bank of available object and activity classifiers; processing the data captured by the selected sensors using the selected classifiers; constructing, within an artificial intelligence (AI) engine of the process parallelization system, a plurality of expert system rules according to the objects, operators, modifiers, and links identified in the received insight request; processing the output from the selected classifier using rules of the expert system; Compiling an Insight Data Packet (IDP) using the outputs of the selected classifiers obtained from processing with rules of the expert system; Distributing the IDP to the plurality of client systems of the process parallelization system. A non-transitory computer-readable medium having stored thereon instructions for configuring the processor to:

Images