Continuous integration and development of code in a secure environment

Abstract

The present disclosure relates to techniques for continuous integration and continuous deployment of source code on a digital health platform. Specifically, aspects relate to receiving source code from a software development system (the source code has been verified according to a first QMS), generating a profile for the source code, classifying the source code into a class of code based on the profile, obtaining a second QMS associated with the class of code, verifying the source code according to the second QMS, and once the source code has been verified, running a build process to generate an executable program from the source code, encapsulating the executable program in a container, and deploying the container on one or more nodes of the software platform. The first QMS is adapted from the perspective of the software development system, while the second QMS is adapted from the perspective of the software platform.

Description

[Technical Field]

[0001] Priority claims This application claims the benefit of and priority to U.S. Provisional Patent Application No. 63 / 111,991, filed November 10, 2020, which is incorporated herein by reference in its entirety for all purposes.

[0002] Field

[0002] The present disclosure relates to digital and personalized healthcare, and more particularly to techniques for continuous integration and continuous deployment of source code in digital health platforms. [Background technology]

[0003] background

[0003] Continuous integration (CI) is a software development technique in which team members and developers test their adjustments as they make changes to the code underlying an application. CI accelerates the release process by finding and fixing bugs earlier in the development cycle and promotes collaboration among developers. Continuous deployment (CD) is the process of getting new software builds to users as quickly as possible. It is the natural next step beyond CI and is an approach used to minimize risk associated with releasing software and new features. To meet the increasing demand for faster software releases and improved quality and security, software development teams often turn to continuous development pipelines to streamline their processes. Adopting continuous integration and continuous deployment (CI / CD) techniques enables on-demand software adaptation, improving time to market, automated testing, security, and user satisfaction.

[0004]

[0004] Data-driven technology solutions are being developed in the healthcare field to enable more personalized care while reducing costs. As the healthcare industry transitions to an on-demand deployment system for personalized healthcare services and solutions, healthcare providers are turning to developers to help them innovate solutions faster through automation and streamlining of software development and service management processes. To support healthcare providers and services, developers are turning to distributed computing environments (e.g., cloud computing) as the standard for healthcare information technology infrastructure. This is seen as a low-cost way to develop the complex infrastructure required to support continuous software development pipelines and deployment within a service model (e.g., analytics as a service (AaaS)). While distributed computing environments like cloud computing offer many benefits to healthcare providers, they also present unique challenges regarding privacy and security due to their functionality that differs from traditional storage and information sharing solutions. For example, because users access data through an internet connection, compliance with government regulations (e.g., Health Insurance Portability and Accountability Act (HIPAA), "good practice" quality guidelines and regulations (GxP), General Data Protection Regulation (GDPR)) poses unique challenges for healthcare providers considering cloud solutions that support continuous software development pipelines and deployment. Therefore, advances are needed for compliant software development platforms built to ensure the confidentiality, availability, and integrity of protected healthcare information. Summary of the Invention

[0005] In various embodiments, a computer-implemented method is provided that includes: receiving, in a continuous integration and continuous deployment (CI / CD) system, source code from a software development system, the source code having been previously verified according to a first quality management system (QMS) associated with the software development system, the first QMS defining a first set of requirements for verifying the source code; generating, by the CI / CD system, a profile of the source code, the profile including identifying characteristics of the source code and characteristics of data operated on by the code, and constructing the profile using the characteristics of the source code and the characteristics of the data operated on by the source code; and converting, by the CI / CD system, the source code into a code clone based on the profile. classifying the source code into a class; obtaining, by the CI / CD system, a second QMS associated with the class of code, the second QMS defining a second set of requirements for validating the class of code; validating, by the CI / CD system, the source code in accordance with the second QMS, including performing code analysis to determine compliance or non-compliance of the source code with the second set of requirements; once the source code is validated, running, by the CI / CD system, a build process to generate an executable program from the source code; encapsulating, by the CI / CD system, the executable program in a container; and deploying, by the CI / CD system, the container on one or more nodes of the software platform.

[0006]

[0006] In some embodiments, the software development system is located remotely from the CI / CD system via a network connection, and a first set of requirements differs from a second set of requirements, and the first set of requirements is defined to determine whether the source code complies with the intended use, operates as intended to implement the intended use, and meets a basic level of security.

[0007]

[0007] In some embodiments, characteristics of the source code are identified by analyzing source code comments and ranges of the source code, the characteristics of the source code including one or more programming languages ​​used to write the source code, the intended use of the source code, and the environment in which the source code is intended to run, and characteristics of the data operated on by the source code include the type and format of data input to the source code and the type and format of data generated by the source code.

[0008]

[0008] In some embodiments, the classifying includes: using the profile to query a data structure that stores a plurality of profiles associated with various classes of code; identifying from the plurality of profiles a likely profile having a predetermined number of characteristics that match characteristics of the source code and characteristics of the data operated on by the source code; and classifying the source code as a class of code associated with the likely profile.

[0009]

[0009] In some embodiments, classifying includes using a classifier that takes the profile as input to generate an estimate of the class of the code based on characteristics of the source code and characteristics of the data operated on by the source code.

[0010]

[0010] In some embodiments, a second set of requirements is defined to determine whether the source code operates as intended to implement the intended use, meets an enhanced level of security, meets requirements imposed by international, national, and / or local regulations, meets data privacy considerations, and meets performance requirements specific to the container or software platform environment.

[0011]

[0011] In some embodiments, the method further includes: configuring, by the CI / CD system, a first pipeline of tools to perform code analysis in accordance with a second QMS, the code analysis including static analysis, and configuring the first pipeline of tools includes: identifying a set of static analysis tools capable of analyzing the source code to satisfy a set of requirements; determining a static test protocol that executes the set of static analysis tools to satisfy a second set of requirements; and provisioning the set of static analysis tools within the first pipeline of tools in accordance with the static test protocol.

[0012] In some embodiments, the method further includes: configuring, by the CI / CD system, a second pipeline of tools to perform code analysis in accordance with a second QMS, the code analysis including dynamic analysis, and configuring the second pipeline of tools includes: identifying a set of dynamic analysis tools capable of analyzing the source code to satisfy the second set of requirements; dynamic determining a dynamic test protocol for executing the set of analysis tools; and provisioning the set of dynamic analysis tools within a second pipeline of tools in accordance with the dynamic test protocol.

[0013]

[0013] In some embodiments, static analysis determines errors, high-risk components, unexpected features, open source features, security vulnerabilities, and changes in source code from version to version, and dynamic analysis determines basic functionality, localized features, performance metrics, and security vulnerabilities.

[0014]

[0014] In some embodiments, determining the open source features includes: identifying the open source features in the source code by the CI / CD system; determining, by the CI / CD system, permissions or licenses associated with the open source features; and compiling, by the CI / CD system, the permissions or licenses associated with the open source features into a data structure.

[0015]

[0015] In some embodiments, the method further includes: identifying, by the CI / CD system, a version of the source code based on static analysis; generating, by the CI / CD system, a new executable program version that includes the source code based on the identified version of the source code; and launching and visualizing, by the CI / CD system, the new executable program version and / or managing the old executable program version that includes the source code.

[0016]

[0016] In some embodiments, the method further includes creating, by the CI / CD system, one or more nodes on the software platform and deploying the executable program in the container.

[0017]

[0017] In some embodiments, the method further includes, when the source code is invalid, providing a notification by the CI / CD system to the software development system that the source code is invalid, the notification including information about one or more reasons why the source code is invalid.

[0018] In some embodiments, a second set of requirements is defined taking into account the intended use of the source code in or as a medical device.

[0019] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium containing instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0020]

[0020] The terms and expressions which have been employed are used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions to exclude any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0021]

[0021] The present disclosure is described in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0022] [Figure 1] 1 illustrates a digital health platform that provides data-driven technology solutions, according to various embodiments. [Figure 2] 1 illustrates a CI / CD system in accordance with various embodiments. [Figure 3] FIG. 1 illustrates a swimlane diagram illustrating the process of code integration, development, and deployment in a digital health platform, according to various embodiments. [Figure 4] 1 illustrates a flow diagram of a process for integrating, developing, and deploying code, according to various embodiments. [Figure 5] 1 illustrates an exemplary pipeline of identified tools and a static test protocol for executing the tools, according to various embodiments. [Figure 6] 1 illustrates an exemplary pipeline of identified tools and a dynamic testing protocol for executing the tools, according to various embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0023]

[0028] In the accompanying drawings, similar components and / or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between the similar components. When only a first reference label is used in this specification, the description is applicable to any of the similar components having the same first reference label, regardless of the second reference label.

[0024] I. Overview

[0029] This disclosure describes techniques for CI / CD of source code on a digital health platform. More specifically, embodiments of the disclosure provide techniques for verifying and deploying various classes of code (e.g., software as a medical device) according to a quality control system that defines a set of requirements for verifying the various classes of source code.

[0025]

[0030] A Quality Management System (QMS) is a set of interrelated or interacting elements, such as policies, objectives, procedures, processes, and resources, established individually or collectively to guide an organization. In the context of this disclosure, organizations engaged in data-driven technology solutions on digital health platforms should establish, implement, monitor, and maintain a QMS that helps ensure they meet the needs of consumers and other stakeholders within the statutory and regulatory requirements associated with their software products, services, or systems. This involves the validation and verification of code to check that a software product, service, or system meets requirements and achieves its intended purpose.

[0026]

[0031] Requirements can be needs or expectations of a system or its software. Requirements can reflect explicit or implied consumer needs and can be market-based, contractual, or statutory, as well as internal to an organization. Requirements can be of various types (e.g., design requirements, functional requirements, implementation requirements, interface requirements, performance requirements, physical requirements). Software requirements are typically derived from system requirements that are aspects of the system functionality assigned to the software. Software requirements are typically written in functional terms and are defined, refined, and updated as the development project progresses.

[0027]

[0032] A specification is a document that states requirements. It may reference or include drawings, patterns, and other related documents, and typically specifies the means or criteria for verifying conformance to the requirements. There are various types of specifications, including system requirements specifications, software requirements specifications, software design specifications, software test specifications, and software integration specifications, all of which are design outputs that establish "specified requirements" and allow for various forms of validation.

[0028]

[0033] Software validation provides objective evidence that the design output at a particular phase of the software development life cycle meets all requirements specified for that phase. Software validation examines the consistency, completeness, and accuracy of the software under development and its supporting documentation, supporting the subsequent conclusion that the software is verified. Software testing is one of many validation activities aimed at ensuring that the output of software development meets the input requirements. Other validation activities include various static and dynamic analyses, code and documentation inspections, walkthroughs, and other techniques.

[0029]

[0034] Software validation is the part of the design verification of a completed software product, service, or system that involves verifying, through inspection or the presentation of objective evidence, that the software specifications meet user needs and intended uses and that specific requirements implemented through the software are consistently met. In practice, software validation activities may occur both during and at the end of the software development lifecycle to ensure that all requirements have been met. Because software is typically part of a larger hardware system, software validation typically includes evidence that all software requirements are correctly and completely implemented and are traceable to system requirements. A conclusion that software is valid relies heavily on comprehensive software testing, inspection, analysis, and other validation tasks performed at each stage of the software development lifecycle. Testing of software product, service, or system functionality in simulated environments or at user sites is typically included as a component of the overall design verification program for the software product, service, or system.

[0030]

[0035] The software development life cycle (SDLC) is a process used in the software industry to design, develop, and test high-quality software according to the requirements defined in a QMS. One of the key tasks in the SDLC is the validation and verification of software source code. As used in this document, "source code" means any complete, executable description of a software system. It is therefore interpreted to include software versions originally written in various programming languages, machine code, or executable graphical representations of the system. When considering data-driven technology solutions in regulated environments such as healthcare, a robust approach to risk-based SDLC code review requires a clearly defined process, the use of qualified infrastructure, the proper design and deployment of software, qualified personnel, and strict change and version control. With the increasing adoption of cloud-based applications and services, cloud computing solutions such as AaaS and Software as a Service (SaaS) offer many benefits, including improved cost-effectiveness, ease of deployment, and a flexible, scalable platform. However, ensuring data integrity and data privacy in cloud computing solutions requires regulated entities to take an informed, proactive approach to planning and maintaining data controls after they are implemented in data-driven technology solutions on cloud provider sites. This involves incorporating a proactive approach for data integrity and data privacy into SDLC code reviews, which can be challenging because it often requires the validation and verification of software source code not developed by the regulated entity. For example, developers may create false statements about the identity of a programming language to directly conceal the true identity of the programming language and avoid proper code scanning. Alternatively, developers may change file extensions to indirectly conceal the true identity of a programming language and avoid proper code scanning.

[0031]

[0036] To address these limitations and issues, the disclosed technique for regulated entity source code validation and deployment utilizes a customized approach to the SDLC. This technique is intended to decouple the developer's portion of the SDLC (in some cases, a CI-based approach developed in accordance with the developer's QMS) from the regulated entity's portion of the SDLC (a customized CI / CD-based approach developed in accordance with the regulated entity's own QMS). This allows the developer to perform n iterations of their portion of the SDLC before ultimately releasing a single version of source code, which then feeds into the regulated entity's portion of the SDLC and is customized to address issues specific to the regulated entity's digital health platform, such as data integrity and data privacy. If the source code fails validation within the regulated entity's portion of the SDLC, the developer is notified and can return to fix the issue that caused the validation failure and perform any number of additional iterations of their portion of the SDLC, with the ultimate goal of deploying the source code to the digital health platform.

[0032]

[0037] An exemplary embodiment of the present disclosure relates to a method, the method including receiving, in a CI / CD system, source code from a software development system, the source code having been verified according to a first QMS associated with the software development system, the first QMS defining a first set of requirements for verifying the source code. The method further includes generating, by the CI / CD system, a profile of the source code. The generating includes identifying characteristics of the source code and characteristics of data operated on by the code, and constructing the profile using the characteristics of the source code and the characteristics of the data operated on by the source code. The method further includes classifying, by the CI / CD system, the source code into a class of code based on the profile, and obtaining, by the CI / CD system, a second QMS associated with the class of code, the second QMS defining a second set of requirements for verifying the class of code. The method further includes verifying, by the CI / CD system, the source code according to the second QMS, the verifying including performing code analysis to determine compliance or non-compliance of the source code with the second set of requirements. The method further includes, once the source code is verified, running, by the CI / CD system, a build process to generate an executable program from the source code, encapsulating, by the CI / CD system, the executable program in a container, and deploying, by the CI / CD system, the container on one or more nodes of the software platform.

[0033] II. Digital Health Platform

[0038] FIG. 1 is a simplified diagram of a digital health platform 100 for providing data-driven technology solutions according to various embodiments. In the illustrated embodiment, the digital health platform 100 includes client computing devices 105 coupled to a cloud-based infrastructure 110 via network(s) 115, including a network gateway 120 and a network mesh 125. The infrastructure 110 is adapted to run services or software applications in service pods 130 using resources provided in a deployment ring 135 by a cloud service provider 140 using various hardware and cloud infrastructures (e.g., private or on-premise cloud infrastructure and public cloud infrastructure). These services or software applications may be provided to users of the client computing devices 105 as web-based or cloud services, such as under an AaaS or SaaS model. Several providers offer cloud services, such as Amazon, Google, and Oracle. The term cloud service is generally used to refer to services made available to users on demand over a communications network, such as the Internet, by a service provider's system (e.g., infrastructure 110), such as a government-regulated entity. Thus, consumers can use cloud services offered by service providers without separately purchasing licenses, support, or hardware and software resources to support the services. For example, a cloud service provider's system can host one or more programs, and users can use the one or more programs on demand over the Internet without purchasing infrastructure resources to run the programs. Cloud services are designed to provide easy and scalable access to applications, resources, and services.

[0034]

[0039] In some cases, a user (e.g., a software or service consumer) operating a client computing device 105 utilizes one or more client applications to consume software products, services, or systems provided by the various components 145 of the infrastructure 110. In other examples, a user (e.g., a developer) operating a client computing device 105 utilizes one or more client applications to upload source code for software products, services, or systems provided by the various components 145 of the infrastructure 110. The components 145 include software components that may be executed by one or more processors, hardware components, or combinations thereof. It should be understood that a variety of different system configurations are possible and may differ from that illustrated for the digital health platform 100. Thus, the embodiment illustrated in FIG. 1 is an example of a distributed computing environment for implementing a digital health platform and is not intended to be limiting.

[0035]

[0040] Client computing devices 105 include various types of computing systems, such as portable handheld devices, general-purpose computers such as personal computers and laptops, workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors and other sensing devices, etc. These computing devices may run various types and versions of software applications and operating systems (e.g., Microsoft Windows®, Apple Macintosh®, UNIX® or UNIX-like operating systems, Linux or Linux-like operating systems, e.g., Google Chrome™ OS), including various mobile operating systems (e.g., Microsoft Windows Mobile®, iOS®, Windows Phone®, Android™, BlackBerry®, Palm OS®). Portable handheld devices may include mobile phones, smartphones (e.g., iPhone®), tablets (e.g., iPad®), personal digital assistants (PDAs), etc. Wearable devices may include Fitbit Versa™ smartwatches, virtual reality (VR) or augmented reality (AR) systems such as Magic Leap 1® and Oculus®, and other devices. Gaming systems may include various handheld gaming consoles, Internet-enabled gaming consoles (e.g., Microsoft Xbox® gaming consoles (with or without Kinect® gesture input devices), Sony PlayStation® systems, various gaming systems offered by Nintendo®, and others). Client device 105 may run various applications, such as various Internet-related apps, communication apps (e.g., email apps, short message service (SMS) apps), and may use various communication protocols.

[0036]

[0041] The network(s) 115 may be any type of network known to those skilled in the art that is capable of supporting data communications using any of a variety of available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk®, etc. By way of example only, the network(s) 115 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating under the Institute of Electrical and Electronics Engineers (IEEE) 1002.11 protocol suite, Bluetooth®, and / or any other wireless protocol), and any combination of these or other networks.

[0037]

[0042] The network gateway 120 is a network node that forms a secure passageway between two or more networks 115 operating on the same or different protocols. The network gateway 120 can provide network security using one or more of the following technologies: firewalls that monitor incoming and outgoing network traffic; virtual private networks that provide private and secure communication channels; security scans that identify security flaws in the network(s); access managers for authentication and authorization services; etc. The network gateway 120 routes network traffic using routers and service connectors (e.g., using a service subscription business model) that manage access to various software products, services, or systems. The network mesh 125 is a local network topology in which the infrastructure 110 (e.g., bridges, switches, and other infrastructure devices) connect directly, dynamically, and non-hierarchically to as many other nodes as possible and cooperate with each other to efficiently route data between devices and nodes. The network mesh 125 manages connectivity using one or more technologies, such as load balancing, product, service, or system discovery, network access, routing, and peering, traffic mirroring, etc. The network(s) 115 , network gateway 120 , and network mesh 125 work in combination to manage all data flowing in and out of infrastructure 110 .

[0038]

[0043] Components 145 may include one or more general-purpose computers, specialized server computers (including, by way of example, PC (personal computer) servers, application-specific servers, midrange servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable configuration and / or combination of computers or systems operating individually or in combination to provide resources, data, services, or programs to client computing devices 105 over network(s) 115. Components 145 may also include other computing architectures involving virtualization, such as one or more virtual machines running virtual operating systems, or one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices. In various embodiments, components 145 are adapted to run one or more services or software applications that provide the functionality described in this disclosure.

[0039]

[0044] Component 145 also includes one or more data repositories. These data repositories may be used to store data and other information in various embodiments. For example, one or more data repositories may be used to store information for providing data-driven technology solutions, such as Software as a Medical Device (SAMD), and for testing and deploying source code to implement the data-driven technology solutions. Data repositories may reside in various locations. For example, a data repository used by a component may be local to the component or remote from the component and communicate with the component via a network-based or dedicated connection. Data repositories may be of various types. In some embodiments, a data repository used by a component may be a database, such as a centralized database, a distributed database, a NoSQL database, a relational database, or the like. One or more of these databases may be adapted to enable data to be stored, updated, and retrieved from the database in response to SQL-formatted commands. In some embodiments, one or more of the databases may be used by an application to store application data. The data repositories used by the applications can be of various types, such as, for example, a key-value store repository, an object store repository, or a general storage repository backed by a file system.

[0040]

[0045] Components 145 also include computing nodes adapted to run one or more programs, such as services or software applications (e.g., services or software applications offered as web-based or cloud services, or applications for implementing a CI / CD system) that provide the functionality described in this disclosure. Each node is a representation of a single machine, optionally implemented within a cluster of nodes. A single machine can be a physical machine (e.g., a server in a data center) or a virtual machine hosted on a cloud provider such as Amazon Web Services™ (AWS) that has a set of available CPU and RAM resources. In a cluster, nodes pool resources to form a more powerful machine. When one or more programs are deployed to a cluster, the cluster intelligently handles work distribution to the individual nodes. If any nodes are added or removed, the cluster can dispatch work as needed. It is not important to the one or more programs or infrastructure 110 which individual machines are actually running the code.

[0041]

[0046] One or more programs to be deployed on one or more clusters are packaged as containers. Containers are a widely accepted standard, allowing various images to be defined for deploying one or more programs on the infrastructure 110. Containerization allows the infrastructure 110 to create self-contained execution environments. Any program and all of its dependencies can be bundled into a single file and shared across the infrastructure 110. Containers can be created programmatically, enabling a powerful, fully automated CI / CD pipeline for testing and deploying code on the infrastructure 110. Containers are wrapped into higher-level constructs known as pods 130. Containers within the same pod 130 can share the same resources and local network. In some cases, containers can communicate with other containers in the same pod 130 as if they were on the same machine, while maintaining a degree of isolation from others. Pods 130 are used as the unit of replication in the infrastructure 110. If a program or resource becomes overwhelmed and a single pod 130 instance cannot handle the load, the infrastructure 110 can be configured to deploy new replicas of the pod 130 across the cluster as needed. Even when not under heavy load, it may be beneficial to have multiple copies of pod 130 running at any one time in a production system to enable load balancing and fault tolerance. One or more instances of pod 130 are provisioned on cloud infrastructure systems provided by one or more cloud service providers 140.

[0042]

[0047] The cloud infrastructure system provided by one or more cloud service providers 140 includes infrastructure resources utilized to facilitate the provisioning of one or more instances of pods 130 that support various cloud services provided by infrastructure 110. To facilitate efficient use of these resources for provisioning one or more instances of pods 130, the resources may be bundled into resource sets or resource modules (also referred to as “deployment rings 135”). Each resource module or deployment ring 135 may include a pre-integrated and optimized combination of one or more types of resources. In certain examples, different deployment rings 135 may be pre-provisioned for different types of cloud services. For example, a first set of deployment rings 135 may be provisioned for SAMD services, and a second set of deployment rings 135 may be provisioned for data analytics services, which may include a different combination of resources than the deployment rings 135 in the first set of deployment rings 135. In some cloud services, resources allocated for provisioning services may be shared between services.

[0043]

[0048] The digital health platform 100 further includes one or more kernels 150. The kernel 150 is adapted to run on each cloud infrastructure system provided by one or more cloud service providers 140. The kernel 150 is a cluster manager that provides resource allocation and isolation across distributed applications or frameworks across the digital health platform 100. The kernel 150 provides an application programming interface (API) to one or more programs for orchestration of services and software, including resource management and scheduling. The architecture of the kernel 150 includes agent nodes for executing tasks, a master node for sending tasks to the agent nodes, a zookeeper for selecting and addressing master nodes, and a framework for collaborating with the master nodes to schedule tasks to the agent nodes.

[0044]

[0049] The digital health platform 100 further includes a CI / CD system 155. The CI / CD system 155 is implemented within a cloud infrastructure system and enables the digital health platform 100 to frequently update, test, and deliver changes within the source code of a software product, service, or system. As described in more detail herein, in healthcare, there are government regulations regarding data security (e.g., data integrity and data privacy) that software must comply with. The CI / CD system 155 can include these policy regulations in the code and automatically track, verify, and reconfigure compliance. In one example of SAMD, data storage locations, server access controls, and activity logging can be included in the source code to help protect and manage user data throughout the use of the software. Additionally, encryption and password-protected operations can be included during continuous integration. Continuous delivery can also use security and monitoring tools to track user activity and detect errors that could lead to security threats.

[0045]

[0050] The CI / CD system 155 can also be used to provision machine learning models. Machine learning models are initially trained using a dataset. However, over time, the model may drift or the data may change, creating the need to update the machine learning model. When a machine learning model runs within a software application, code associated with the software application can include triggers for when to retrain the machine learning model. For example, the code can include instructions to retrain the machine learning model at predetermined time intervals, when new training data becomes available, or when the machine learning model's performance is determined to fall below a threshold. Furthermore, software developers can explore variations in model architecture and hyperparameters in a test environment based on monitoring the machine learning model's performance in a production environment or based on estimated improvements for model optimization. The CI / CD system 155 facilitates building, testing, and deploying the machine learning model to a production environment once it is determined that the model meets performance requirements.

[0046] III. CI / CD System

[0051] FIG. 2 illustrates a schematic diagram of a CI / CD system 200 (e.g., CI / CD system 155 described with respect to FIG. 1 ) for verifying and deploying various classes of code, according to various embodiments. In the illustrated embodiment, CI / CD system 200 includes actors 205, CI components 210, and CD components 215. Actors 205 may include software developers, code reviewers, quality engineers, release managers, and / or service providers, such as government-regulated entities. Actors 205 upload source code to CI / CD system 200 for validation, verification, and deployment through one or more client devices (e.g., client device 105 described with respect to FIG. 1 ). As illustrated, actors 205 and the portions of the SDLC associated with actors 205 are physically separated from the regulated entity's portions of the SDLC (CI components 210 and CD components 215). This allows actor 205 to perform n iterations of his or her portion of the SDLC before ultimately releasing a single version of source code that is input into CI component 210 and CD component 215 and customized to address concerns specific to the regulated entity's digital health platform, such as data integrity and data privacy. If the source code fails validation within CI component 210 and CD component 215, actor 205 is notified and can go back to fix the issue that caused the validation failure and perform any number of additional iterations of his or her portion of the SDLC, with the ultimate goal of deploying the source code to a software platform (e.g., government-regulated entity's digital health platform 100, as described in FIG. 1).

[0047]

[0052] CI component 210 includes a documentation system 220 for maintaining various QMSs, code profiles, and code review records, a task management system 225 for profiling source code and orchestrating code review tasks, a code hosting platform 230 for version control and collaboration, and a code review system 235 for performing static analysis of source code. When submitting source code, actor 205 builds the source code into a unique set of code changes and then pushes the unique set of code changes to CI component 210. According to various aspects of the present disclosure, source code uploaded to CI component 210 has typically already been verified according to a QMS associated with the software development system (e.g., a third-party developer's software development system). Task management system 225 is adapted to automatically identify the class of source code being uploaded to CI component 210 so that CI component 210 can efficiently and accurately validate and verify the source code. Specifically, the task management system 225 is adapted to generate a profile of the source code to identify a class of source code, identify a QMS specifically configured for the identified class of source code, and configure an optimal set of code review tasks according to a set of requirements defined by the identified QMS for validating and verifying the source code.

[0048]

[0053] It should be understood that there are many different types or classes of source code and software that may be released by actors 205 to the software platform's CI component 210 and CD component 215. Accordingly, task management system 225 is designed to validate and verify all of the different types or classes of source code and software released to CI / CD system 200. Furthermore, the intent of actors 205 may not always be known to or consistent with the intent of the software platform. For example, some of actors 205 may attempt to attack the software platform by uploading malicious code to hijack secure sessions, steal data, extract sensitive personal data, obtain user security credentials, crash the network or the software platform, etc. For example, some of actors 205 may create false statements about the identity of source code to directly conceal the true identity of the source code and avoid proper code scanning. Alternatively, some of actors 205 may change file extensions to indirectly conceal the true identity of the source code and avoid proper code scanning. Thus, task management system 225 is adapted to classify received source code regardless of the actor's intent and identify a QMS that defines a set of requirements for validating the source code that is specific to the class of source code. The set of requirements can be defined to determine whether the source code: operates as intended to fulfill the intended use; meets an enhanced level of security; meets requirements imposed by international, national, and / or local regulations; meets data privacy considerations; and meets performance requirements specific to container and software platform environments. The identified QMS enables CI / CD system 200 to validate, verify, and deploy the source code in an efficient, effective, and secure manner.

[0049]

[0054] The task management system 225 obtains the identified QMS from the document system 220 and transfers the identified QMS, the optimal set of code review tasks, and the source code to the code hosting platform 230. The code review system 235 then performs code validation according to the optimal set of code review tasks. The optimal set of code review tasks defines a set of tests to be performed on the source code. The code review system 235 includes a static analysis tool that performs the set of tests to analyze the source code. The code review system 235 determines compliance or non-compliance of the source code with a set of requirements based on the results of the set of tests performed on the source code. The code review system 235 then determines the validity of the source code based on its compliance or non-compliance with the set of requirements. Once the compliance and / or validity of the source code is confirmed, the code hosting platform 230 releases the source code to the CD component 215 for code construction and further testing (e.g., static analysis, dynamic analysis, and / or white sourcing). Releasing involves transferring the source code and, optionally, the identified optimal set of QMS and code review tasks to a code building tool or engine 240.

[0050]

[0055] The CD components 210 include a code build tool or engine 240 for managing code builds, a code build system 245 for constructing code, a code review system 250 for performing static and dynamic analysis of source code, an artifactory 255 for storing build artifacts, and a deployment manager 260 for encapsulating executable programs / artifacts into containers and deploying the container executable programs / artifacts onto a software platform. The code build tool or engine 240 is adapted to automate the creation of software builds and related processes, including compiling computer source code into binary code, packaging the binary code, and running automated tests. The code build system 245 is adapted to perform full builds, which are builds from scratch, or incremental builds, which use the last build state maintained internally by the code build engine 240 to perform builds based on changes in the project since the last build state. A full build or incremental build includes fetching code from the code build engine 240 or code repository, compiling the code (including checking dependencies), running automated tests using the code review system 250, linking libraries, code, files, etc. as appropriate, and, after validation, building and saving the artifacts (including saving the build log).

[0051]

[0056] The code review system 250 performs code validation according to an optimal set of code review tasks. The optimal set of code review tasks defines a set of tests to be run against the source code (the set of tests can be the same or similar to those run against the source code before the build, different tests, or a combination thereof). The code review system 250 includes static and dynamic analysis tools that run the set of tests to analyze the source code. The code review system 250 determines compliance or non-compliance of the source code with a set of requirements based on the results of the set of tests run against the source code. In some cases, the code review system 250 determines the validity of the source code based on compliance or non-compliance with the set of requirements. Once the source code is validated and / or verified, the code build system 245 builds and stores artifacts in the artifactory 255 along with the developed software. An artifact is one of many types of tangible by-products generated during software development. Some artifacts (e.g., use cases, class diagrams, and other Unified Modeling Language (UML) models, requirements, and design documents) help describe the functionality, architecture, and design of software. In certain cases, artifacts also include source code compiled for testing, as executables are required to execute the test plan.

[0052]

[0057] The deployment manager 260 encapsulates or packages software into one or more containers using one or more customized pre-built images and wraps one or more containers into a pod. The deployment manager 260 also creates a node or cluster of nodes supported by a resource or set of resource modules (e.g., deployment ring 135 described with reference to FIG. 1 ) within one or more distributed computing environments 265 (e.g., cloud computing). The deployment layer 260 launches pods on the node or cluster of nodes directly or indirectly (through one or more deployment abstraction layers). The deployment layer is adapted to monitor the pods, determine the number of replicas of the pod running on the node or cluster of nodes, and add / replace pods as needed. The deployment manager 260 registers software or services supported by the software (e.g., publishes the software or services) and opens secure communication channels for users to access the software or services.

[0053] IV. Technology for CI / CD of source code on digital health platforms

[0058] 3-4 illustrate processes and operations for verifying and deploying source code. Note that particular embodiments may be described as a process that is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. While the flowcharts describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the diagrams. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0054]

[0059] The processes illustrated in FIGS. 3-4 may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of hardware, or a combination thereof. The software may be stored in memory (e.g., a memory device, a non-transitory computer-readable storage medium). The specific process steps illustrated in FIGS. 3-4 are not intended to be limiting. Other sequences of steps may be performed according to alternative embodiments. For example, the steps outlined above may be performed in a different order in alternative embodiments. Furthermore, individual steps illustrated in FIGS. 3-4 may include multiple sub-steps, which may be performed in various sequences as appropriate for the individual steps. Moreover, additional steps may be added or deleted depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0055]

[0060] FIG. 3 illustrates a process 300 for validating and deploying source code on a software platform using a CI / CD system. The process illustrated in flow diagram 300 is implemented by the architecture, systems, and technologies illustrated in FIGS. 1 and 2. In step 305, a release manager of a software development system creates a release of the source code. The release constitutes a partial or fully functional version of the source code or software that the release manager intends to build and deploy on the software platform. As illustrated, there may be multiple release managers associated with one or more software development systems (i.e., actors). Each release manager may create one or more releases of the source code for build and deployment. The software development system releases the source code to the CI / CD system of the software platform (e.g., the government-regulated entity's digital health platform 100, as described in FIG. 1) for validation, verification, and deployment.

[0056]

[0061] In step 310, the CI of the software development system associated with a given release manager retrieves the source code for deployment and executes a build process to construct an executable program. The build process is performed to convert the source code into a standalone form or artifact (e.g., an efficient executable program) that can run on a computing device. The build process may involve a combination of several activities that vary depending on the programming language and operating system, but the basic concept will remain the same. The source code and its artifacts are fetched from a repository, compiled while checking dependencies, automated tests are run to verify operability, files, libraries, and code are linked together, and if the tests are successful, the artifact representing the executable program is finalized and saved, and a build log is recorded.

[0057]

[0062] In step 315, the CI of the software development system tests and verifies the source code and executable program according to the software development system's QMS. For example, each software development system may establish a software life cycle model for verifying developed software within the framework of a QMS appropriate for the release manager's product and organization. The QMS defines a set of requirements for verifying the source code. For example, the set of requirements may be defined to determine whether the source code conforms to its intended use, operates as intended to perform its intended use, and meets a basic level of security. For each software life cycle activity, there are actionable tasks that support the conclusion that the software is verified. However, the tasks performed, the order in which they are performed, and the iterations and timing of their execution depend on the specific software life cycle model selected and the safety risks associated with each software application identified by the software development system. Each task for verifying software typically begins with requirements. For example, for each software function, actors should be able to list the requirements that describe that function. Furthermore, for each requirement, actors should have a test plan to verify that the function functions as required, and a recordkeeping system to record evidence that the plan was executed and the test results.

[0058]

[0063] In step 320, the CI of the software development system checks the quality of the source code. Code quality checks include checks for reliability (e.g., determining whether the code executes as intended with minimal defects), maintainability (e.g., follows a consistent structure / style, is easy to understand, is well documented, etc.), and testability (e.g., how well the software supports validation / validation testing efforts). In step 325, a determination is made as to whether the source code and executable program are valid and / or of sufficient quality. If the source code and executable program are invalid (do not meet all or some of the requirements of the QMS) and / or are of low quality (as defined and measured by the software development system in accordance with the QMS), the CI of the software development system forwards the source code to the development team to fix the defects and / or quality of the source code. In contrast, if the source code and executable programs are valid (meet all or some of the requirements of the QMS) and / or are of good quality (as defined and measured by the software development system in accordance with the QMS), the software development system's CI will release the source code to the software platform.

[0059]

[0064] In step 330, the software platform's CI / CD system receives the source code and automatically identifies the QMS to be used to validate, verify, and deploy the source code. For example, the entities that control the software development system and their associated portions of the SDLC are separated from the entities that control the CI / CD system and their associated portions of the SDLC. As a result, the entities that control the software development system can perform n iterations of their portion of the SDLC (e.g., steps 305-320) and ultimately release a single version of the source code, which is received in step 330. This physical separation between the software development system and the CI / CD system allows the CI / CD system to be customized to address challenges specific to the software platform, such as data integrity and data privacy, while providing an additional layer of privacy and security between the software development system and the CI / CD system.

[0060]

[0065] In some cases, the entity that controls the CI / CD system may be the same as or different from the entity that controls the software development system. In either case, the source code has already been verified in accordance with the software development system's QMS. However, it should be understood that there are many different types or classes of source code and software that may be released to a software platform from various software development systems. For example, one software development system may be developing a SAMD, another software development system may be developing machine learning models for image processing, another software development system may be developing software to be implemented in a medical device, and another software development system may be developing software used to verify the medical device or the SAMD, etc. Therefore, the software platform's QMS is designed to validate and verify all of the different types or classes of source code and software released to the CI / CD system.

[0061]

[0066] Furthermore, the intent of actors 205 may not always be known to or coincide with the intent of the software platform. For example, some actors 205 may attempt to attack the software platform by uploading malicious code to hijack secure sessions, steal data, exploit sensitive personal data, obtain user security credentials, crash the network or the software platform, etc. For example, some actors 205 may create false statements about the identity of source code to directly conceal the true identity of the source code and avoid proper code scanning. Alternatively, some actors 205 may change file extensions to indirectly conceal the true identity of the source code and avoid proper code scanning. Therefore, the CI / CD system is adapted to classify received source code regardless of the actor's intent and identify a QMS that defines a set of requirements for validating the source code that is specific to the class of source code. A set of requirements can be defined to determine whether source code: operates as intended to fulfill its intended use; meets an enhanced level of security; satisfies requirements imposed by international, national, and / or local regulations; satisfies data privacy considerations; and meets performance requirements specific to container and software platform environments. The identified QMS enables the CI / CD system to validate, verify, and deploy source code in an efficient, effective, and secure manner.

[0062]

[0067] Once the QMS is identified, the software platform's CI / CD system performs quality and compliance checks on the source code. Quality and compliance checks include both the build process, which builds an executable program, and the validation and verification of the source code and executable program. The build process, as described in 310, is performed to convert the source code into a standalone form or artifact (e.g., an efficient executable program) that can run on a computing device. Validation is performed to ensure that the source code or executable program is built according to requirements. In other words, to ensure that the source code or executable program meets those specified requirements. Verification is performed to ensure that the source code or executable program actually meets user needs and that the specifications were correct in the first place. In other words, to demonstrate that the source code or executable program will meet additional requirements, including the specified needs, when deployed in the intended environment. Performing quality and compliance checks involves several subprocesses, such as developing test plans, executing test plans, and recording the execution of test plans. Source code test plans are automatically developed by the CI / CD system in accordance with the QMS. For example, a CI / CD system would be aware of all types of validation and verification tasks that can be performed by the system, including static and dynamic analysis to test source code functionality, and the CI / CD system would be aware of all functional requirements that need to be evaluated for source code verification via the identified QMS. Thus, the CI / CD system would develop a test plan that includes the tasks to be performed, the order in which the tasks will be performed, the iterations and timing of task performance, and the pipeline of tools to test the source code functionality.

[0063]

[0068] Test plans are executed by the CI / CD system using a pipeline of automated code review tools. Generally, tools can be classified into one of two groups based on the type of analysis they perform: dynamic analysis and static analysis. Dynamic analysis is performed on the executable code of a compiled program and checks only user-specific behavior. That is, only the code executed during testing (e.g., object code) is checked. Dynamic analyzers can provide users with information about memory leaks, program performance, call stacks, etc. Static analysis allows checking program code (e.g., source code) before the tested program is executed. Compilers typically perform static analysis during the compilation process. However, large-scale, real-world projects often require static analysis to be performed throughout development to ensure that source code meets additional requirements, including reliability, maintainability, readability, testability, portability, and vulnerability. Many different analysis techniques are implemented in dynamic and static analyzers, such as abstract syntax tree (AST) walker analysis, data flow analysis, and path-sensitive data flow analysis. These different analysis techniques will be organized and executed according to the test plan.

[0064]

[0069] Various analysis techniques are evaluated to determine the source code's validity (quality and compliance or non-compliance) against the requirements defined within the QMS. In other words, it is ensured that the source code or software meets its specified requirements. In some cases, the quality and compliance or non-compliance determination is an all-or-nothing determination, e.g., all requirements must be met for the source code to be of adequate quality and compliance. In other cases, the compliance or non-compliance determination is flexible, e.g., a certain percentage or threshold number of requirements must be met for the source code to be of adequate quality and compliance. If the various analysis techniques demonstrate that the source code is of adequate quality and compliance, the validity of the source code will be determined according to the requirements defined within the QMS (e.g., user needs or the correctness of the original specification). In contrast, if the various analysis techniques demonstrate that the source code is not of sufficient quality and compliance, a notification may be sent to actors and / or administrators of the software platform. The notification identifies the source code and the reason for the validation failure. The results of the test plan execution and validation are recorded in a manner that maintains traceability between the functional requirements, the test plans against those functional requirements, and the evidence that the test plans were executed. This record is retained in a repository for future use, for example, to support government regulatory reviews.

[0065]

[0070] The various analysis techniques are evaluated to determine the validity of the source code according to the requirements defined within the QMS (e.g., user needs and the correctness of the specifications in the first place). In some cases, the validity determination is an all-or-nothing decision, e.g., the source code must meet all requirements to be considered valid. In other cases, the determination of compliance or non-compliance is flexible, e.g., the source code must meet a certain percentage or threshold number of requirements to be considered valid. If the various analysis techniques demonstrate that the source code is valid, the source code proceeds to block 320 for a white sourcing determination. In contrast, if the various analysis techniques demonstrate that the source code is not valid, a notification can be sent to actors and / or administrators of the software platform. The notification identifies the source code and the reason for the verification failure. The execution of the test plan and the verification results are recorded in a manner that maintains traceability between the functional requirements, the test plans for the functional requirements, and evidence supporting the execution of the test plans. This record is retained in a repository for future use, for example, to support government regulatory reviews.

[0066]

[0071] The set of validation and verification requirements may include one or more of the following: operate as intended to carry out the intended use; satisfy requirements imposed by international, national, and / or regional regulations; satisfy data integrity and privacy considerations; satisfy performance requirements specific to the container or software platform environment; and satisfy extended levels of security. Intended use requirements assess whether the overall functionality of various features is implemented as intended to implement the overall intended use of the source code or software (e.g., image processing for tumor detection). International, national, and / or regional regulatory requirements assess whether government regulations specific to the software platform (such as national security, cybersecurity, software malfunction, and data privacy) are met. In some cases, these regulatory requirements are based on the international, national, and / or regional location where the source code and / or data used by the source code are deployed and used. Data integrity and privacy requirements assess whether protection from unauthorized access and data corruption is met. In some cases, data integrity and privacy requirements are based on the international, national, and / or regional location where the source code and / or data used by the source code are deployed and used. Performance requirements specific to containers and software platforms are assessed to determine whether asset and data isolation is met.

[0067]

[0072] The enhanced security level requirements evaluate whether security features are functioning to prevent or minimize vulnerabilities specific to the software platform. For example, a software development system's QMS may define a baseline level of security that all of its source code or software must meet to address common vulnerabilities such as SQL injection, OS command vulnerabilities, buffer overflows, uncontrolled form strings, and integer overflows. However, a software platform's QMS may define enhanced security levels that all source code or software deployed on the software platform must meet to address specific vulnerabilities specific to the software platform. For example, enhanced security levels may include requirements for maintaining separation between tenants, applications, clusters, etc. to prevent attackers from accessing assets or data from one application from another; requirements for complete and verifiable data deletion; requirements for API disclosure; requirements for data recovery in the event of loss or theft of encryption keys; restrictions on insider abuse of authenticated access; and restrictions on the use of unauthorized cloud services.

[0068]

[0073] In step 335, the CI / CD system performs white sourcing of the source code. White sourcing identifies open source components, including dependencies, within the source code or software and compiles the open source components into a data structure. In some cases, white sourcing also: (i) identifies permissions or licenses associated with the open source components, (ii) identifies vulnerabilities resulting from the open source components (e.g., vulnerabilities arising from permissions or licenses associated with the open source components), (iii) enforces license policies throughout the SDLC, and (iv) compiles the permissions or licenses, vulnerabilities, and associated open source components into a data structure. In some cases, white sourcing prioritizes vulnerabilities based on whether source code exploits them to stratify risk management.

[0069]

[0074] In step 340, a determination is made as to whether the source code and executable programs are valid and / or contain vulnerabilities due to open source components. If the source code and executable programs are invalid (do not meet all or part of the requirements of the QMS) and / or contain vulnerabilities due to open source components (as defined and determined by the software platform in accordance with the QMS), the CI / CD system transfers the source code to the development team to fix the defects and / or vulnerabilities in the source code. On the other hand, if the source code and executable programs are valid (meet all or part of the requirements of the QMS) and / or do not contain vulnerabilities due to open source components (as defined and determined by the software platform in accordance with the QMS), the CI / CD system proceeds with the deployment process of the software's source code.

[0070]

[0075] In step 345, the CI / CD system performs version control and creates and stores artifacts in an artifact repository. Version control tracks all changes to source code in a code repository. For example, a version control system might maintain a complete long-term change history for every file. This represents all changes made by many individuals or actors during the life of the source code or software. Changes include file creation and deletion as well as edits to its contents. Version control systems also allow for branching and merging of multiple workstreams related to source code or software. Version control systems further enable traceability of each change made to source code or software and can connect actors with annotations about the reason and intent for changes, such as bugs and errors. Artifacts are created and stored in an artifact repository, or universal repository. An artifact repository manages the end-to-end artifact lifecycle and provides consistency for CI / CD systems and workflows while supporting a variety of source code and software package management systems. An artifact repository is both the source of artifacts needed for the build process and the target for deploying artifacts produced by the build process.

[0071]

[0076] In step 350, the executable program is encapsulated in a container and deployed as a pod instance on one or more nodes of the software platform. Upon deployment of the executable program, a deployment notification of the pod instance of the executable program is provided to the CI / CD system. In block 355, the CI / CD system receives the deployment notification of the pod instance and forwards the notification to the release manager. In step 360, the process loops back in the release manager to allow the development team to fix defects and / or quality issues in the source code.

[0072]

[0077] FIG. 4 illustrates a process 400 for verifying and deploying source code on a software platform using a CI / CD system. The process illustrated in flow diagram 400 is implemented by the architecture, systems, and technologies illustrated in FIGS. 1 and 2. In step 405, the source code verified by the first QMS is accessed from a software development system. The source code is accessed from the software platform's CI / CD system. In some cases, the software development system is located remotely from the CI / CD system via a network connection. The first QMS defines a set of requirements for verifying the source code. The first set of requirements is adapted to determine one or more of the following: whether the source code conforms to its intended use, whether it operates as intended to perform its intended use, and whether it meets a basic level of security. The first QMS is customized to address a wide range of challenges software developers face in developing quality source code, such as ensuring that software developers meet the needs of consumers and other stakeholders associated with their software products, services, or systems. This involves validating and verifying the code to check that the software product, service, or system meets a first set of requirements.

[0073]

[0078] At step 410, a profile of the source code is generated. The profile is generated by: (i) identifying characteristics of the source code and characteristics of the data operated on by the code; and (ii) constructing the profile using the characteristics of the source code and characteristics of the data operated on by the code. Characteristics of the source code may be identified by analyzing a range of source code comments on the source code and by analyzing the technology used by the code (e.g., Java or a mobile platform). Characteristics of the source code may include one or more programming languages ​​used to write the source code, the intended use of the source code, the environment in which the source code is intended to run, the environment in which the source code was developed (e.g., country of origin), etc. Characteristics of the data operated on by the source code may include the type and format of data input to the source code and the type and format of data generated by the source code. For example, the type and format may include whether the data is streaming data, model training data, integrity and privacy data, data compiled from SAMD, historical data, or archived data, etc.

[0074]

[0079] At step 415, the source code is classified into a class of code based on the profile. Classification may include using the profile to query a data structure that stores profiles associated with various classes of code. Likely profiles may be identified from profiles having a predetermined number of characteristics that match characteristics of the source code and characteristics of the data operated on by the source code. For example, a query may be used to identify a set of likely profiles from profiles having a predetermined number of characteristics that match characteristics of the source code and characteristics of the data operated on by the source code. The set of profiles may be sorted using one or more sorting characteristics, such as a number of matching characteristics, a relevance value, or a strength of match value. Profiles at the top or bottom of the list of the set of profiles may be identified as likely profiles. The source code may be classified as a class of code associated with the likely profile. Alternatively, classification may use a classifier that takes the profile as input to generate an estimate of the class of code based on characteristics of the source code and characteristics of the data operated on by the source code. Advantageously, the classification process of steps 410 and 415 enables many different types or classes of source code and software to be verified and deployed. Furthermore, because the source code is classified regardless of the class assigned by the software developer / software development system, a second QMS for the class of code classified in step 415 is obtained, enabling the CI / CD system to validate, verify, and deploy the source code in an efficient, effective, and safe manner.

[0075]

[0080] In step 420, a second QMS is obtained for the class of code determined in step 515. The second QMS defines a second set of requirements for validating the class of code. In some cases, the first set of requirements differs from the second set of requirements. The second set of requirements may include determining whether the source code: operates as intended to fulfill its intended use (e.g., in or as a medical device); meets an increased level of security; meets requirements imposed by international, national, and / or local regulations; meets data privacy considerations; and meets performance requirements specific to the container or software platform environment. In certain cases, the second set of requirements is defined taking into account that the source code's intended use is in or as a medical device. The second QMS is customized to address more specific challenges faced by software providers, such as government-regulated entities, in providing quality source code, for example, to ensure that the software provider meets the needs of consumers and other stakeholders within statutory and regulatory requirements related to software products, services, or systems. This includes validating and verifying the code to check that the software product, service, or system meets a second set of requirements. This physical separation between the software development system and the CI / CD system allows the CI / CD system and second QMS to be customized to address the specific challenges unique to such software platforms.

[0076]

[0081] In step 425, a first pipeline of tools is configured to perform code analysis in accordance with the second QMS. The code analysis includes static analysis. Configuring the first pipeline of tools includes: (i) identifying a set of static analysis tools capable of analyzing the source code to satisfy the second set of requirements, (ii) determining a static test protocol for executing the set of static analysis tools to satisfy the second set of requirements, and (iii) provisioning the set of static analysis tools within the tools of the first pipeline in accordance with the static test protocol. FIG. 5 shows an example of an identified pipeline of tools and a static test protocol for executing the tools. The static test protocol defines parameters, an execution order of the set of static analysis tools, and criteria for determining compliance or non-compliance of the source code with the second set of requirements. The static test protocol includes first analyzing and detecting the programming language of the source code and, based on the detected programming language, performing one or more of the following: finding code and comments, finding empty lines, finding unused code, finding duplicate code, and comparing code for differences. The static testing then includes performing one or more of the following: detecting errors in the code, determining the code's conformance to coding standards, detecting semantic and syntactic errors, and detecting domain-related coding errors. The static testing protocol then includes performing one or more of the following: identifying security vulnerabilities, detecting open source components, and determining whether the code complies with various third-party regulations (e.g., government regulations such as HIPAA). Finally, the static testing protocol compiles the output of the various tools and generates metrics and reports to ultimately determine whether the source code has been verified in accordance with the second QMS. It should be understood that the tools and static testing protocol shown in FIG. 5 are merely exemplary, and that any number and types of static analysis tools may be used, and various static testing protocols may be configured to run the tools and perform code analysis in accordance with the second QMS.

[0077]

[0082] In step 430, a second pipeline of tools is configured to perform code analysis in accordance with the second QMS. The code analysis includes dynamic analysis. Configuring the second pipeline of tools includes: (i) identifying a set of dynamic analysis tools capable of analyzing source code to satisfy the second set of requirements, (ii) determining a dynamic test protocol for executing the set of dynamic analysis tools to satisfy the second set of requirements, and (iii) provisioning the set of dynamic analysis tools within the tools of the second pipeline in accordance with the dynamic test protocol. FIG. 6 illustrates an example of an identified pipeline of tools and a dynamic test protocol for executing the tools. The dynamic test protocol may define parameters, an execution order of the set of dynamic analysis tools, and criteria for determining compliance or non-compliance of the source code with the second set of requirements. The dynamic test protocol includes first executing the code or executable program at runtime and, based on the execution, performing one or more of the following: functional testing, testing of critical functions, testing of functions of specific functionality, testing and / or localization of compatibility and integration with other systems, and regression analysis. The dynamic testing then includes performing one or more of the following: performance testing, load and stress testing, and volume and durability testing. The dynamic testing protocol then includes identifying security vulnerabilities and determining the usability of the code or executable program. Finally, the dynamic testing protocol compiles the output of the various tools and generates metrics and reports to ultimately determine whether the source code has been verified in accordance with the second QMS. It should be understood that the tools and dynamic testing protocol shown in FIG. 6 are merely exemplary, and that any number and type of dynamic analysis tools may be used, and various dynamic testing protocols may be configured to execute the tools and perform code analysis in accordance with the second QMS.

[0078]

[0083] In step 435, the source code is verified according to the second QMS. The verification includes performing code analysis to determine the source code's compliance or non-compliance with a second set of requirements. The code analysis is evaluated to determine the source code's validity (quality and compliance or non-compliance) against the second set of requirements defined within the second QMS. If the source code is demonstrated to be of adequate quality and compliance by various analysis techniques, the validity of the source code is determined. The code analysis is evaluated to determine the validity of the source code according to the second set of requirements defined within the second QMS (e.g., user needs, or that the specification was correct in the first place).

[0079]

[0084] The code analysis may include static analysis and / or dynamic analysis performed according to the first pipeline of tools and / or the second pipeline of tools configured in steps 425 and 430, respectively. The static analysis may determine errors, high-risk components, unexpected features, open source features, security vulnerabilities, and changes from version to version in the source code. The dynamic analysis may determine base functionality, localized features, performance metrics, and security vulnerabilities. In some cases, the static analysis determines errors, high-risk components, unexpected features, open source features, security vulnerabilities, and changes from version to version in the source code. In some cases, the dynamic analysis determines base functionality, localized features, performance metrics, and security vulnerabilities. In some cases, determining the open source features includes performing a white sourcing process. The white sourcing process includes: (i) identifying open source features in the source code, (ii) determining permissions or licenses associated with the open source features, and (iii) compiling the permissions or licenses and associated open source features into a data structure.

[0080]

[0085] If the source code is invalidated, notification of the invalidation is provided to the software development system in step 440. The notification may include information regarding one or more reasons for the invalidation.

[0081]

[0086] Once the source code is verified, a build process is performed in step 445 to generate an executable program from the source code and perform version control of the executable program. The version control may include identifying a version of the source code based on static analysis and generating a version of the executable program that includes the source code based on the identified version of the source code. The version control may further include launching and visualizing new executable program versions and / or managing older executable program versions that include the source code.

[0082]

[0087] At step 450, the executable program is deployed on the software platform. Deploying includes: (i) encapsulating the executable program in a container, and (ii) deploying the container to one or more nodes of the software platform. In some cases, deploying further includes creating one or more nodes on the software platform to host the executable program in the container. In some cases, deploying further includes creating a pod instance that wraps the container. The pod instance is deployed on one or more nodes of the software platform.

[0083] V. Additional Considerations

[0088] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium containing instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0084]

[0089] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0085]

[0090] The following description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of preferred exemplary embodiments will provide those skilled in the art with an enabling description for implementing various embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the appended claims.

[0086]

[0091] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments can be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

Claims

1. 1. A computer-implemented method comprising: In a continuous integration and continuous deployment (CI / CD) system, receiving source code from a software development system, the source code having been verified according to a first quality management system (QMS) associated with the software development system, the first QMS defining a first set of requirements for verifying the source code; generating a profile of source code by a CI / CD system, identifying characteristics of source code and characteristics of data operated on by said source code; constructing a profile using characteristics of the source code and characteristics of the data operated on by said source code; generating a profile of the source code, Classifying source code into classes of code based on profiles by the CI / CD system; obtaining, by the CI / CD system, a second QMS associated with the class of code, the second QMS defining a second set of requirements for verifying the class of code; verifying, by the CI / CD system, the source code in accordance with a second QMS, including performing code analysis to determine compliance or non-compliance of the source code with a second set of requirements; Once the source code is verified, the CI / CD system executes a build process to generate an executable program from the source code; Encapsulating executable programs into containers by a CI / CD system; and Deploying the container on one or more nodes of the software platform by the CI / CD system. A method comprising:

2. a software development system located remotely from the CI / CD system via a network connection; The computer-implemented method of claim 1 , wherein the first set of requirements is different from the second set of requirements.

3. 10. The computer-implemented method of claim 1, wherein the first set of requirements is defined to determine whether the source code conforms to the intended use, operates as intended to carry out the intended use, and meets a basic level of security.

4. Characteristics of the source code are identified by analyzing source code comments and ranges of the source code; Characteristics of the source code include the programming language(s) used to write the source code, the intended use of the source code, and the environment in which the source code is intended to run; 4. The computer-implemented method of claim 1, wherein the characteristics of the data operated on by the source code include the type and format of data input to the source code and the type and format of data generated by the source code.

5. To classify, Using the profile to query a data structure that stores multiple profiles associated with different classes of code; identifying a likely profile from the plurality of profiles that has a predetermined number of characteristics that match characteristics of the source code and characteristics of the data operated on by the source code; and classifying said source code into classes of code associated with likely profiles; The computer-implemented method of any one of claims 1 to 4, comprising:

6. 5. The computer-implemented method of claim 1, wherein classifying comprises using a classifier that takes the profile as input to generate an estimate of the class of the code based on characteristics of the source code and characteristics of the data operated on by the source code.

7. 7. The computer-implemented method of claim 1, wherein the second set of requirements is defined to determine whether the source code operates as intended to perform the intended use, meets an enhanced level of security, meets requirements imposed by international, national, and / or local regulations, meets data privacy considerations, and meets performance requirements specific to container and software platform environments.

8. The method further includes configuring, by the CI / CD system, a first pipeline of tools and performing code analysis according to a second QMS, wherein the code analysis includes static analysis, and configuring the first pipeline of tools further comprises: Identifying a set of static analysis tools that can analyze source code to meet a set of requirements; determining a static testing protocol that executes a set of static analysis tools to meet a second set of requirements; and The computer-implemented method of any one of claims 1 to 7, comprising provisioning a set of static analysis tools in a first pipeline of tools according to a static testing protocol.

9. The method further includes configuring, by the CI / CD system, a second pipeline of tools and performing code analysis according to a second QMS, wherein the code analysis includes dynamic analysis, and configuring the second pipeline of tools further comprises: identifying a set of dynamic analysis tools capable of analyzing the source code to meet a second set of requirements; determining a dynamic testing protocol that executes a set of dynamic analysis tools to meet a second set of requirements; and 10. The computer-implemented method of claim 8, further comprising provisioning a set of dynamic analysis tools in a second pipeline of tools according to a dynamic testing protocol.

10. 10. The computer-implemented method of claim 9, wherein static analysis determines errors, high-risk components, unexpected features, open source features, security vulnerabilities, and changes in source code from version to version, and dynamic analysis determines basic functionality, localized features, performance metrics, and security vulnerabilities.

11. Deciding on open source features is key. Identifying open source features in source code through a CI / CD system; Determining, via the CI / CD system, any permissions or licenses associated with the open source features; and Compiling permissions or licenses associated with open source features into a data structure by the CI / CD system. The computer-implemented method of claim 10, comprising:

12. Identifying the version of the source code based on static analysis by the CI / CD system; generating, by the CI / CD system, a new executable program version that includes the source code based on the identified version of the source code; and Activating and visualizing new executable program versions and / or managing older executable program versions, including the source code, through a CI / CD system. The computer-implemented method of any one of claims 1 to 11, further comprising:

13. The computer-implemented method of any one of claims 1 to 11, further comprising creating, by the CI / CD system, one or more nodes on the software platform to host executable programs in containers.

14. 14. The computer-implemented method of claim 1, further comprising, when the source code is invalid, providing, by the CI / CD system, a notification to the software development system that the source code is invalid, the notification including information regarding one or more reasons why the source code is invalid.

15. The computer-implemented method of any one of claims 1 to 14, wherein the second set of requirements is defined taking into account the intended use of the source code in or as a medical device.

16. one or more data processors in a continuous integration and continuous deployment (CI / CD) system; and Non-transitory computer-readable storage medium containing instructions wherein the instructions, when executed on the one or more data processors, cause the one or more data processors to: receiving source code from a software development system, the source code having been verified in accordance with a first quality management system (QMS) associated with the software development system, the first QMS defining a first set of requirements for verifying the source code; generating a profile of source code, identifying characteristics of source code and characteristics of data operated on by said source code; constructing a profile using characteristics of the source code and characteristics of the data operated on by said source code; generating a profile of the source code, Classifying source code into classes of code based on profiles; obtaining a second QMS associated with the class of code, the second QMS defining a second set of requirements for verifying the class of code; verifying the source code in accordance with a second QMS, including performing code analysis to determine compliance or non-compliance of the source code with a second set of requirements; Once the source code is verified, running a build process to generate an executable program from the source code; Encapsulating an executable program in a container; and Deploying the container on one or more nodes of the software platform A system that performs an operation including:

17. a software development system located remotely from the CI / CD system via a network connection; 17. The system of claim 16, wherein the first set of requirements is different from the second set of requirements.

18. 17. The system of claim 16, wherein the first set of requirements is defined to determine whether the source code conforms to the intended use, operates as intended to carry out the intended use, and meets a basic level of security.

19. Characteristics of the source code are identified by analyzing source code comments and ranges of the source code; Characteristics of the source code include the programming language(s) used to write the source code, the intended use of the source code, and the environment in which the source code is intended to run; 19. The system of any one of claims 16 to 18, wherein the characteristics of the data operated on by the source code include the type and format of data input to the source code and the type and format of data generated by the source code.

20. To classify, Using the profile to query a data structure that stores multiple profiles associated with different classes of code; identifying a likely profile from the plurality of profiles that has a predetermined number of characteristics that match characteristics of the source code and characteristics of the data operated on by the source code; and classifying said source code into classes of code associated with likely profiles; The system according to any one of claims 16 to 19, comprising:

21. 20. The system of any one of claims 16 to 19, wherein classifying comprises using a classifier that takes the profile as input to generate an estimate of the class of the code based on characteristics of the source code and characteristics of the data operated on by the source code.

22. 22. The system of any one of claims 16 to 21, wherein the second set of requirements is defined to determine whether the source code operates as intended to perform the intended use, meets an enhanced level of security, meets requirements imposed by international, national, and / or local regulations, meets data privacy considerations, and meets performance requirements specific to container and software platform environments.

23. The operation is and further comprising configuring, by the CI / CD system, a first pipeline of tools and performing code analysis according to the second QMS, wherein the code analysis includes static analysis, and configuring the first pipeline of tools further comprises: Identifying a set of static analysis tools that can analyze source code to meet a set of requirements; determining a static testing protocol that executes a set of static analysis tools to meet a second set of requirements; and Provisioning a set of static analysis tools within a first pipeline of tools according to a static testing protocol The system of any one of claims 16 to 22, comprising:

24. The operation is and further comprising configuring, by the CI / CD system, a second pipeline of tools and performing code analysis according to the second QMS, wherein the code analysis includes dynamic analysis, and configuring the second pipeline of tools further comprises: identifying a set of dynamic analysis tools capable of analyzing the source code to meet a second set of requirements; determining a dynamic testing protocol that executes a set of dynamic analysis tools to meet a second set of requirements; and Provisioning a set of dynamic analysis tools within a second pipeline of tools in accordance with a dynamic testing protocol.

24. The system of claim 23, comprising:

25. 25. The system of claim 24, wherein static analysis determines errors, high-risk components, unexpected features, open source features, security vulnerabilities, and changes in source code from version to version, and dynamic analysis determines basic functionality, localized features, performance metrics, and security vulnerabilities.

26. Deciding on open source features is key. Identifying open source features in source code through a CI / CD system; Determining, via the CI / CD system, any permissions or licenses associated with the open source features; and Compiling permissions or licenses associated with open source features into a data structure by the CI / CD system.

26. The system of claim 25, comprising:

27. The operation is Identifying the version of the source code based on static analysis by the CI / CD system; generating, by the CI / CD system, a new executable program version that includes the source code based on the identified version of the source code; and Activating and visualizing new executable program versions and / or managing older executable program versions, including the source code, through a CI / CD system. The system of any one of claims 16 to 26, further comprising:

28. 27. The system of claim 16, wherein the operations further include creating, by the CI / CD system, one or more nodes on a software platform to host executable programs in containers.

29. 29. The system of claim 16, wherein the operations further include, when the source code is invalid, providing, by the CI / CD system, a notification to the software development system that the source code is invalid, the notification including information about one or more reasons for the source code being invalid.

30. The system of any one of claims 16 to 29, wherein the second set of requirements is defined taking into account the intended use of the source code in or as a medical device.

31. 1. A computer program product for causing one or more data processors in a continuous integration and continuous deployment (CI / CD) system to perform operations, the operations comprising: receiving source code from a software development system, the source code having been verified in accordance with a first quality management system (QMS) associated with the software development system, the first QMS defining a first set of requirements for verifying the source code; generating a profile of source code, identifying characteristics of source code and characteristics of data operated on by said source code; constructing a profile using characteristics of the source code and characteristics of the data operated on by said source code; generating a profile of the source code, Classifying source code into classes of code based on profiles; obtaining a second QMS associated with the class of code, the second QMS defining a second set of requirements for verifying the class of code; verifying the source code in accordance with a second QMS, including performing code analysis to determine compliance or non-compliance of the source code with a second set of requirements; Once the source code is verified, running a build process to generate an executable program from the source code; Encapsulating an executable program in a container; and Deploying the container on one or more nodes of the software platform a computer program comprising:

32. a software development system located remotely from the CI / CD system via a network connection; 32. The computer program of claim 31, wherein the first set of requirements is different from the second set of requirements.

33. 32. The computer program of claim 31 , wherein the first set of requirements is defined to determine whether the source code conforms to the intended use, operates as intended to carry out the intended use, and meets a basic level of security.

34. Characteristics of the source code are identified by analyzing source code comments and ranges of the source code; Characteristics of the source code include the programming language(s) used to write the source code, the intended use of the source code, and the environment in which the source code is intended to run; 34. A computer program as claimed in any one of claims 31 to 33, wherein the characteristics of data operated on by the source code include the type and format of data input to the source code and the type and format of data generated by the source code.

35. To classify, Using the profile to query a data structure that stores multiple profiles associated with different classes of code; identifying a likely profile from the plurality of profiles that has a predetermined number of characteristics that match characteristics of the source code and characteristics of the data operated on by the source code; and classifying said source code into classes of code associated with likely profiles; 35. A computer program according to any one of claims 31 to 34, comprising:

36. 35. The computer program of claim 31, wherein classifying comprises using a classifier that takes the profile as input to generate an estimate of the class of the code based on characteristics of the source code and characteristics of data operated on by the source code.

37. 37. The computer program of any one of claims 31 to 36, wherein the second set of requirements is defined to determine whether the source code operates as intended to perform the intended use, meets an enhanced level of security, meets requirements imposed by international, national, and / or local regulations, meets data privacy considerations, and meets performance requirements specific to container and software platform environments.

38. The operation is and further comprising configuring, by the CI / CD system, a first pipeline of tools and performing code analysis according to the second QMS, wherein the code analysis includes static analysis, and configuring the first pipeline of tools further comprises: Identifying a set of static analysis tools that can analyze source code to meet a set of requirements; determining a static testing protocol that executes a set of static analysis tools to meet a second set of requirements; and Provisioning a set of static analysis tools within a first pipeline of tools according to a static testing protocol 38. A computer program according to any one of claims 31 to 37, comprising:

39. The operation is and further comprising configuring, by the CI / CD system, a second pipeline of tools and performing code analysis according to the second QMS, wherein the code analysis includes dynamic analysis, and configuring the second pipeline of tools further comprises: identifying a set of dynamic analysis tools capable of analyzing the source code to meet a second set of requirements; determining a dynamic testing protocol that executes a set of dynamic analysis tools to meet a second set of requirements; and Provisioning a set of dynamic analysis tools within a second pipeline of tools in accordance with a dynamic testing protocol.

39. The computer program of claim 38, comprising:

40. 40. The computer program product of claim 39, wherein static analysis determines errors, high-risk components, unexpected features, open source features, security vulnerabilities, and changes in source code from version to version, and dynamic analysis determines basic functionality, localized features, performance metrics, and security vulnerabilities.

41. Deciding on open source features is key. Identifying open source features in source code through a CI / CD system; Determining, via the CI / CD system, any permissions or licenses associated with the open source features; and Compiling permissions or licenses associated with open source features into a data structure by the CI / CD system.

41. The computer program of claim 40, comprising:

42. The operation is Identifying the version of the source code based on static analysis by the CI / CD system; generating, by the CI / CD system, a new executable program version that includes the source code based on the identified version of the source code; and Activating and visualizing new executable program versions and / or managing older executable program versions, including the source code, through a CI / CD system.

42. The computer program of claim 31, further comprising:

43. 42. The computer program product of claim 31, wherein the operations further include creating, by a CI / CD system, one or more nodes on a software platform to host executable programs in containers.

44. 44. The computer program product of claim 31, wherein the operations further include, when the source code is invalid, providing, by the CI / CD system, a notification to the software development system that the source code is invalid, the notification including information about one or more reasons for the source code being invalid.

45. A computer program according to any one of claims 31 to 44, wherein the second set of requirements is defined taking into account that the intended use of the source code is in or as a medical device.

46. A non-transitory machine-readable storage medium having the computer program according to any one of claims 31 to 45 recorded thereon.

Images