Facilitating automated security analytics

Abstract

The invention relates to facilitating automated safety analysis. In various examples, systems and methods related to facilitating automated security analytics are disclosed. In particular, a security analysis associated with a software product or a portion thereof may be performed to identify any potential security risk associated with the software product. To perform code security analysis, various types of code security data may be analyzed to comprehensively analyze any potential security risk associated with the product. For example, various stages of product development may be analyzed to facilitate prevention of security disruption associated with the product. In implementation, design analysis, design-to-code analysis, and / or code analysis may be performed to identify potential security risks. Performing these various analyses enables robust and comprehensive security assessment of products from design to implementation.

Description

Background Technology

[0001] Security reviews, such as offensive security reviews, are typically conducted to examine the functionality of software products in an effort to identify any potential security issues. Typically, security reviews occur in the final stages of product development. For example, an offensive security review might be performed after product functionality has been developed but before the product has been released, to analyze security from an attacker's perspective. Such offensive security reviews can facilitate cybersecurity strategies, specifically helping to identify potential security threats and / or defend against them before they can be exploited by attackers.

[0002] In conventional implementations, security reviews are typically performed manually by the review team to identify security vulnerabilities or issues. However, manually performing security reviews is both time-consuming and error-prone. For example, an offensive security review of a specific product can take weeks, with various computing resources being used to facilitate manual review.

[0003] In some cases, static code analysis tools can be used to facilitate a more efficient security review process. Static code analysis tools typically examine source code, bytecode, and / or compiled binaries without executing the program. Specifically, they can systematically scan code to find patterns and common issues that may lead to security problems. However, security reviews using static code analysis tools often only detect a subset of security issues. As an example, static code analysis tools rely on predefined rules and patterns to identify problems. In cases where vulnerabilities do not match any predetermined patterns, security issues may not be detected. In this respect, static code analysis tools often miss vulnerabilities or misidentify problems for complex code structures and patterns. Furthermore, static code analysis tools may omit the analysis of deeply nested or rarely used code paths, potentially missing associated security vulnerabilities. As another example, static code analysis tools are often not designed to identify highly specialized or context-dependent vulnerabilities requiring deep security knowledge. As yet another example, static code analysis tools analyze code and are often language-specific. Thus, static code analysis tools are often general in application and do not consider the design or requirements associated with the product.

[0004] Furthermore, static code analysis tools can lead to false positives and / or false negatives. For example, to avoid missing potential issues, static code analysis tools often use conservative rules, which can result in false positives, thus consuming computational resources for further evaluation of code execution. Since static code analysis tools do not execute code, they cannot verify potential security issues, leading to false positives as well. Static analysis tools may also miss vulnerabilities by failing to analyze various coding patterns or contexts in which vulnerabilities might occur, increasing the cost and computational resources required to subsequently provide solutions. In addition to the computational resource utilization required to resolve undetected security issues, the attacked computer is often also affected, resulting in unnecessary computational resource utilization such as disk space, I / O operations, CPU and memory usage, and power consumption. Summary of the Invention

[0005] Various embodiments of this disclosure relate to facilitating automated security analysis. Systems and methods are disclosed for performing security analyses associated with software products or portions thereof in an efficient and effective manner. To perform code security analysis, various types of code security data can be analyzed to perform a comprehensive analysis. Utilizing various types of code security data enables a more comprehensive analysis of any potential security risks associated with the product. For example, different stages of product development can be analyzed to facilitate the prevention of security breaches associated with the product. In operation, one or more machine learning models can be used to perform design analysis, design-to-code analysis, and / or code analysis to identify potential security risks. In embodiments, based on the identification of one or more potential security risks, security risk mitigation solutions can be generated automatically to reduce or mitigate security risks.

[0006] Compared to conventional implementations, performing automated security analysis associated with different aspects of product development enables more robust analysis, thereby increasing the opportunity to identify potential security risks associated with the product. In addition to increasing the identification of potential security risks and resulting in more secure products, computational resource utilization is also reduced because addressing undetected security issues requires fewer resources or resource utilization can be spread across the entire product development lifecycle. Furthermore, identifying potential security risks associated with the product avoids unnecessary computational resource utilization associated with computers that might otherwise be vulnerable to attacks that lead to disk space utilization, I / O operations, CPU and memory usage, power consumption, etc. Moreover, as described in this paper, using AI technologies to perform security analysis enables the more efficient and effective identification of any potential security risks associated with the product. Attached Figure Description

[0007] The following describes in detail, with reference to the accompanying drawings, existing systems and methods for facilitating automated security analysis, wherein:

[0008] Figure 1 Example network environments according to some embodiments of this disclosure are provided;

[0009] Figure 2 The illustrations depict example implementations of some embodiments of the present disclosure for facilitating the management of automated security analysis via a security analysis manager;

[0010] Figure 3A Example designs for threat identification tips are provided according to some embodiments of this disclosure;

[0011] Figure 3B Example design-threat analysis tips are provided according to some embodiments of this disclosure;

[0012] Figure 3C Example designs for identifying security requirements are provided according to some embodiments of this disclosure;

[0013] Figure 3D Example design-security requirement analysis tips are provided based on some embodiments of this disclosure;

[0014] Figure 3E Example hints are provided for designing evaluation systems according to some embodiments of this disclosure;

[0015] Figure 3F Example design-code mapping hints are provided according to some embodiments of this disclosure;

[0016] Figure 3G Example design-code analysis tips are provided according to some embodiments of this disclosure;

[0017] Figure 3H Sample code interface hints are provided according to some embodiments of this disclosure;

[0018] Figure 3I Sample code analysis tips are provided according to some embodiments of this disclosure;

[0019] Figure 3J Example security analysis reports and tagging statuses are provided according to some embodiments of this disclosure;

[0020] Figure 3K Example hints for analyzing function implementations according to some embodiments of this disclosure are provided;

[0021] Figure 3L Example vulnerability patterns are provided according to some embodiments of this disclosure;

[0022] Figure 3M Example vulnerability analysis reports and tagging statuses are provided according to some embodiments of this disclosure;

[0023] Figure 3N Example solutions for identifying prompts according to some embodiments of this disclosure are provided;

[0024] Figure 4 Example streams for facilitating the management of automated security analysis are provided according to some embodiments of this disclosure;

[0025] Figure 5 Example methods for performing automated security analysis according to some embodiments of this disclosure are provided;

[0026] Figure 6 Example methods for performing automated security analysis according to some embodiments of this disclosure are provided;

[0027] Figure 7 Example methods for performing automated security analysis according to some embodiments of this disclosure are provided;

[0028] Figure 8A This is a block diagram of an example generative language model system applicable to implementing at least some embodiments of the present disclosure;

[0029] Figure 8B This is a block diagram of an example generative language model including a converter encoder-decoder, applicable to implementing at least some embodiments of this disclosure;

[0030] Figure 8C This is a block diagram of an example generative language model including a decoder-only converter architecture suitable for implementing at least some embodiments of this disclosure;

[0031] Figure 9 This is a block diagram of an example computing device applicable to implementing at least some embodiments of the present disclosure; and

[0032] Figure 10 This is a block diagram of an example data center applicable to implementing at least some embodiments of the present disclosure. Detailed Implementation

[0033] Systems and methods related to facilitating automated security analysis are disclosed. Specifically, security analyses associated with software products or parts thereof can be performed to identify any potential security risks associated with the software product. To perform code security analysis, various types of code security data can be analyzed to perform a comprehensive analysis. For example, design data, code data, threat data, security requirement data, proprietary security data, and / or open-source data can be analyzed to identify potential security risks associated with software products or parts thereof. Utilizing various types of code security data enables a more comprehensive analysis of any potential security risks associated with the product. For example, different stages of product development can be analyzed to facilitate the prevention of security breaches associated with the product.

[0034] In implementation, the embodiments described herein can perform design analysis, design-to-code analysis, and / or code analysis to identify potential security risks. Design analysis examines the design associated with the product using various types of data, such as threat data, security requirement data, proprietary security data, and / or open-source data. In this way, design analysis identifies security deficiencies or defects associated with the design of the software product that may pose security risks. Design-to-code analysis typically analyzes the implementation or coding associated with the product's design. For example, design-to-code analysis can identify whether the design is implemented as intended. Where the design is not implemented as intended via code, potential security risks may be exposed, thus identifying these potential security risks. Code analysis typically analyzes programming or implementation problems in the code that may lead to security risks. In this regard, even if the product can be implemented as intended via design, the implementation may still lead to security risks (e.g., via integer overflow, memory corruption, etc.). Performing such various analyses enables a robust and comprehensive security assessment of the product from design to implementation.

[0035] To perform design analysis, design-to-code analysis, and / or code analysis, AI techniques such as Large Language Models (LLM) can be used to facilitate efficient and effective analysis. Specifically, this technology can be used to identify relevant data to be analyzed. For example, to perform design analysis, LLM can be used to identify security risks in relevant portions of the design data to be analyzed. In addition to identifying relevant data, AI techniques can also be used to perform security analysis on relevant data. For example, LLM can be used to analyze relevant design data to identify any security risks associated with that design data.

[0036] In one example, during operation, various code security data are acquired. For example, design data, code data, threat data, security requirement data, proprietary security data, and / or open-source data can be obtained via user devices and / or data sources. Design data typically refers to data indicating the design associated with code and / or software products. Code data typically refers to the code to be analyzed. Threat data typically refers to data indicating security threats. Security requirement data can include any requirements or guidelines generated to provide software security. Proprietary security (PS) data typically refers to information and findings collected within the organization (e.g., data obtained by an offensive security review team during previous security reviews). In this respect, PS data includes examples of security issues found in the code or design of various projects that can be used to improve security across the organization (e.g., the company). Open-source (OS) data can include any data collected from open-source projects, particularly data focused on security issues.

[0037] Based on the obtained code security data, the code security data can be preprocessed to transform the obtained code security data or a portion thereof into a format suitable for analysis. Thus, in some embodiments, the code security data or a portion thereof can be converted into vector data. As an example only, code security data from which LLM can retrieve information can be converted into vector data. In some cases, code security data in text form can be indexed and transformed into vector data. In other cases, code security data in image form (e.g., threat data) can be translated into text and then transformed into vector data. For example, an image can be translated into text using an image-to-text model. Vector data can be stored in a data store (e.g., a vector store).

[0038] Using the acquired data, various analyses can be performed to identify any potential security risks associated with a specific software product or its components. In this regard, design analysis, design-to-code analysis, and / or code analysis associated with the software product can be performed to identify any potential security risks.

[0039] Regarding performing design analysis, software product design can be analyzed to examine the intended functionality and interactions of various components (e.g., within a system) to ensure that the design accurately represents how the system should operate and / or how different components communicate with each other without introducing security risks. In this way, design analysis facilitates the identification of potential security risks during the design phase, independent of any coding or implementation work. Design analysis may include analyzing the product design or portions thereof in relation to various types of code security data, such as threat data, security requirement data, PS data, and / or OS data. In this manner, the product design associated with various code security data can be analyzed efficiently and effectively to identify potential security issues.

[0040] As described, a product design can be analyzed in relation to threats to identify any defects in the design that are associated with identified threats (e.g., existing or anticipated threats). To analyze one or more designs associated with threat data, one or more designs associated with various threats can be analyzed to identify any potential security risks associated with the product design. Potential security risks can be identified where the product design does not include any design data indicating mitigation of corresponding threats.

[0041] To analyze design data associated with threats, relevant design data corresponding to the threats can be identified. As described, in one example, design data is represented via vector data stored in a data store. In this way, for a threat, relevant design data can be identified based on a search of the corresponding vector data. Based on the identified matching design documentation fragments, relevant product designs can be analyzed to determine whether these designs address the corresponding security threat. If the design does not address or mitigate the corresponding security threat, then the threat is identified as a potential security risk. This process of identifying relevant or matching design data associated with a threat and then analyzing that design data can be performed (e.g., via a data store) for various threats (e.g., each threat in a threat set).

[0042] In various embodiments, artificial intelligence techniques, such as machine learning or other technologies, are used to identify and / or analyze relevant parts of the design associated with threats. In this way, an LLM (or VLM or MMLM, etc.) can be used to identify and / or analyze relevant parts of the design. To use an LLM, one or more prompts can be generated for input into the LLM to identify and / or analyze relevant design data associated with threat data. As an example, a first prompt can be generated, and this first prompt can be used to identify relevant design data associated with one or more threats; and a second prompt can be generated, and this second prompt can be used to analyze relevant design data associated with one or more threats (e.g., to analyze security risks).

[0043] It is also possible to analyze the design of a product associated with safety requirements to identify any defects in the design related to the identified safety requirements. In this way, one or more designs associated with various safety requirements can be analyzed to identify one or more potential safety risks associated with the product design. Potential safety risks can be identified where the design does not include any indication of implementing or meeting the corresponding safety requirements.

[0044] In practice, to analyze safety requirement data associated with safety requirements, relevant design data corresponding to those requirements can be analyzed. In one example, design data can be represented as vector data stored in a data store (e.g., vector storage). In this way, for a safety requirement, relevant design data can be identified based on a search of the corresponding vector data. Based on the identified matching design document fragments, the relevant design data can be analyzed to determine whether the product design addresses the corresponding safety requirement. If the design fails to address or meet the corresponding safety requirement, that safety requirement is identified as a potential safety risk.

[0045] In various embodiments, AI technologies, such as machine learning or other techniques, are used to identify and / or analyze relevant portions of the design associated with security requirements. In this way, an LLM can be used to identify and / or analyze relevant portions of the design. To use an LLM, one or more prompts can be generated and input into the LLM to identify and / or analyze relevant design data associated with security requirement data. As an example, a first prompt can be generated, and this first prompt can be used to identify relevant design data associated with one or more security requirements; and a second prompt can be generated, and this second prompt can be used to analyze relevant design data associated with one or more security requirements.

[0046] Regarding design analysis associated with PS and / or OS data, one or more designs associated with various PS and / or OS data can be analyzed to identify potential security risks related to the product design. Potential security risks can be identified where the design data lacks any indication of mitigating security vulnerabilities corresponding to the PS and / or OS data. In this way, the design of a software product, such as that identified via design data, can be analyzed based on PS and / or OS patterns to identify whether the occurrence of that pattern or similar patterns has been mitigated in the design.

[0047] In operation, to analyze design data associated with PS and / or OS modes, relevant design data corresponding to the PS and / or OS modes can be identified. In one example, design data can be represented via vector data stored in a data store. In this way, for PS and / or OS modes, relevant design data can be identified based on a search of the corresponding vector data. Based on the identified matching design document fragments, the relevant design data can be analyzed to determine whether the product design addresses the corresponding PS and / or OS modes. If the design fails to address, mitigate, or achieve the corresponding PS and / or OS modes, the PS and / or OS modes can be identified as potential security risks.

[0048] In various embodiments, AI technologies such as machine learning or other techniques can be used to identify and / or analyze relevant portions of the design associated with PS and / or OS modes. As an example, a machine learning model in the form of an LLM can be used to identify and / or analyze relevant portions of the design associated with PS and / or OS modes. To use an LLM, one or more prompts can be generated and used as input to the LLM to identify and / or analyze relevant design data associated with PS and / or OS data. As an example, a first prompt can be generated, and this first prompt can be used to identify relevant design data associated with one or more PS and / or OS modes; and a second prompt can be generated, and this second prompt can be used to analyze relevant design data associated with one or more PS and / or OS modes.

[0049] For design-to-code analysis, design data can be compared with code data to assess whether the desired design has been captured in the implementation. Potential security risks can be identified where inconsistencies exist between design and code. In various embodiments, design-to-code mapping analysis is performed. In this respect, product design can be mapped to actual code implementations to identify whether developers have accurately interpreted and reflected the design in the code. As an example, for various functions in a function set, a search can be performed (e.g., via query) to identify corresponding design fragments (e.g., in a data store such as a data storage). Thus, design data is mapped to its corresponding actual code implementation. Using LLM, codebases and design documents can be examined to find matching elements. The LLM can then be prompted to search the source code stored in the data store to locate the specific code fragment implementing the functionality. The mapped data can then be analyzed to identify any inconsistencies that may exist or indicate security issues. In this respect, once the design-to-code mapping is established, inconsistencies in each mapping pair are further analyzed. LLM can help identify any deviations or errors in the code that may arise from misunderstandings of the design, thereby ensuring that the implementation aligns with the original design intent.

[0050] For code-to-design matching (e.g., as indicated in a mapping table), the identified design data associated with the code snippet can be analyzed. In this regard, any inconsistencies between the matched code data and the design data can be identified. Inconsistencies indicate that the code and design are not aligned, thus indicating that the code implementation does not conform to the design. Any such inconsistencies can be identified and provided as output data. In some cases, LLM or other AI techniques can be used to perform this type of analysis. In this regard, where matching design data snippets associated with code snippets (e.g., code functions) are identified, the data can be analyzed to evaluate any inconsistencies between the data. In this way, design data and corresponding or matching code data can be analyzed via LLM to identify any design-related inconsistencies or code gaps between the matching data.

[0051] For code analysis, code is analyzed to assess data flows and / or patterns that indicate potential security risks. In this way, security risks can be analyzed even in instances where the design is suited to a product and implemented according to such a suitable design. In various embodiments, code can be analyzed to identify potential security risks based on the programming language or implementation used to create the software product. For example, dangerous patterns (e.g., direct memory access) may arise from code generated based on encoding or programming language.

[0052] In various embodiments, data flow analysis, code analysis using PS data and / or OS data, and / or test coverage analysis can be performed to facilitate code analysis to identify security risk data. Data flow analysis typically refers to analyzing data flows associated with code to identify one or more security risks. Specifically, for data flow analysis, the inputs of source code functions are analyzed and how these inputs are subsequently used in the function logic. In this way, the data flows and interfaces exposed to users are analyzed to identify potential security issues.

[0053] First, to perform data flow analysis, code can be analyzed to identify specific code snippets for further analysis of security issues. In this way, code corresponding to potential user access or interfaces can also be analyzed. In some cases, to identify code snippets, sets of code functions can be analyzed. As an example, a design-code map set can be analyzed to identify code functions that include interfaces exposed to users. In some cases, AI techniques can be used to facilitate the identification of code functions for further evaluation. As an example only, a code function can be accessed (e.g., via a design-code map), and this code function can be used to generate prompts to identify the data input to the function, how the function uses it, and / or the data output by the function.

[0054] Based on the identified functions requiring further analysis (e.g., based on potential interfaces, data flow analysis results, or data exposure to users or other security issues), a function can be analyzed for potential security problems. As an example only, suppose the function's implementation uses untrusted data to perform specific operations, such as memory access. In this case, the data flow associated with the function can be identified as having a dangerous pattern and therefore potentially a security problem. For example, LLM can be used to analyze whether the data flow associated with the function corresponds to or matches a security risk pattern identified as a potential security problem. Security risk patterns can include any known patterns that indicate a security vulnerability. In various embodiments, such a security risk pattern may correspond to specific programming code used to create or develop a product. One example of a security risk pattern might reflect one or more memory corruptions. Another example of a security risk pattern is integer overflow. Integer overflow can occur when an arithmetic operation results in a value exceeding the maximum (or minimum) value that the data type can hold, potentially leading to unexpected values. Other examples of security vulnerabilities that may arise due to the characteristics and / or features of programming languages ​​may include SQL injection, cross-site scripting, cross-site request forgery, command injection, path traversal, buffer overflow, use after free, null pointer dereference, deserialization vulnerabilities, etc.

[0055] In some cases, PS data and / or OS data can be used to perform code analysis. In this scenario, code can be analyzed to identify or determine whether it contains one or more specific security risk patterns identified via PS data and / or OS data. In this case, code functions can be compared to security risk patterns generated via PS data and OS data. Security risk patterns can be generic patterns generated or extracted based on PS data or OS data. In other cases, security risk patterns can be examples of data (e.g., code) containing security risks. In various embodiments, AI techniques such as LLM can be used to determine whether one or more code functions exhibit or match one or more security risk patterns. In this way, similarity between code functions and security risk patterns can be identified via prompts input to the LLM. If similarity or degree of similarity is identified, the security risk pattern can be identified as a match.

[0056] Alternatively or additionally, test coverage analysis can be performed to facilitate code analysis. In this regard, covered code segments (e.g., functions) or one or more corresponding security tests can be identified. Security tests generally refer to any test used to identify or verify security risks. Examples of security tests include unit tests, integration tests, fuzzing, etc. By using these different types of test coverage functions, developers can ensure that software is robust, reliable, and secure. Thus, performing test coverage analysis provides insights into potential security risks associated with the code or parts thereof.

[0057] In this way, various security risk data representing or indicating security risks can be identified by performing design analysis, design-to-code analysis, and / or code analysis. For example, such security risk data can be provided to user devices that request or initiate security analysis related to a product. In this manner, any indication of a security risk or data associated with it can be provided to users, such as those requesting to perform security analysis and / or view such data. Various examples of identifiable security risk data include indications of code locations (e.g., lines) corresponding to potential security risks, code snippets associated with security risks, risk scores indicating the likelihood and / or severity of a potential security risk, explanations or causes related to the severity or likelihood of a security risk, analyses performed to identify potential security risks, and sources used to identify potential security risks (e.g., threat data, security requirements, etc.).

[0058] In some embodiments, security risk solutions associated with security risks can be automatically generated and / or implemented. In this regard, solutions can be generated for security risks identified through design analysis, design-to-code analysis, and / or code analysis (e.g., via LLM). Thus, security risk solutions can be automatically generated to reduce or remediate potential security risks.

[0059] Advantageously, performing security analyses associated with different aspects of product development enables more robust analysis, thereby increasing the chances of identifying potential security risks associated with the product. In addition to the increased identification of potential security risks leading to a more secure product, computational resource utilization is also reduced because addressing undetected security issues requires fewer resources. Furthermore, identifying potential security risks associated with the product avoids unnecessary computational resource utilization associated with computers that might otherwise be vulnerable to attacks that result in disk space utilization, I / O operations, CPU and memory usage, power consumption, etc. Moreover, as described in this paper, using AI technologies to perform security analyses enables the more efficient and effective identification of any potential security risks associated with the product.

[0060] refer to Figure 1 , Figure 1This is an example network environment according to some embodiments of this disclosure. It should be understood that the arrangements and other arrangements described herein are merely illustrative examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used in addition to or as alternatives to the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. The various functions described herein as being performed by entities can be performed by hardware, firmware, and / or software. For example, various functions can be performed using one or more processors that execute instructions stored in one or more memories. For example, in some embodiments, the systems and methods described herein may use one or more generative language models (e.g., such as...). Figures 8A-8C (as described), one or more computing devices or components thereof (e.g., such as) Figure 9 (as described) and / or one or more data centers or components thereof (e.g., such as) Figure 10 To achieve this, as described above.

[0061] Continue to refer to Figure 1 This diagram illustrates a block diagram of an exemplary network environment 100 suitable for implementing the embodiments described herein. Typically, system 100 illustrates an environment suitable for facilitating automated security analysis. The embodiments described herein effectively and efficiently analyze the security associated with code to identify any security risks associated with that code. According to the embodiments described herein, language models such as LLM can be used to perform various aspects of code security analysis in an automated manner. To analyze the code security associated with a product or parts thereof, design analysis, design-to-code analysis, and / or code analysis, as described herein, can be performed automatically.

[0062] In operation, users such as code security auditors can input, provide, or instruct code security data, and based on this input, automatically provide representations or security risk data. Code security data generally refers to any data that can be used to perform code security analysis. Examples of code security data include design data, code data, threat data, security requirement data, PS data, OS data, etc. When analyzing this type of data, any identified security risk data can be provided to users, for example, those initiating code security analysis via user devices. Security risk data generally refers to any data associated with potential security risks identified through code security analysis.

[0063] Network environment 100 includes user equipment 110, security analysis manager 112, and data storage 114. User equipment 110, security analysis manager 112, and data storage 114 can communicate via network 122, which can include any number of networks, such as, for example, local area network (LAN), wide area network (WAN), Internet, cellular network, peer-to-peer (P2P) network, mobile network, or a combination of networks.

[0064] Figure 1 The network environment 100 shown is an example of a suitable network environment and is not intended to impose any limitation on the scope or functionality of the embodiments disclosed throughout this document, nor should the exemplary network environment 100 be construed as having any dependency or requirement relating to any individual component or combination of components illustrated herein. For example, a user device may communicate with the security analysis manager 112 via a mobile network or the Internet, and the security analysis manager 112 may communicate with the data storage 114 via a local area network. Further, although environment 100 is illustrated via a network, one or more components may communicate directly with each other, for example, via HDMI (High Definition Multimedia Interface) and DVI (Digital Video Interface). Alternatively, one or more components may be integrated with each other. For example, at least a portion of the security analysis manager 112 and / or the data storage 114 may be integrated with the user device 110. For example, a portion of the security analysis manager 112 may be integrated with a server communicating with the user device 110, and another portion of the security analysis manager 112 may be integrated with the user device 110.

[0065] User equipment 110 can be any kind of computing device capable of facilitating efficient and effective analysis of code security. For example, in one embodiment, user equipment 110 can be such as the following reference Figure 9 The computing device described is a computing device such as computing device 900. In various embodiments, user equipment 110 may be a personal computer (PC), laptop computer, workstation, mobile computing device, personal digital assistant (PDA), mobile phone, etc.

[0066] User equipment 110 may include one or more processors and one or more computer-readable media. The computer-readable media may include computer-readable instructions executable by one or more processors. The instructions may be executed by one or more applications (such as...) Figure 1Application 120 (as shown) embodies this. One or more applications can generally be any application capable of facilitating the management of automated code security analysis. In some cases, one or more applications (such as application 120) can facilitate automated code security analysis. In some implementations, one or more applications include web applications that can run in a web browser and can be hosted at least partially (e.g., via security analysis manager 112) on a server side. Additionally or alternatively, one or more applications may include dedicated applications. In some cases, the application is integrated into the operating system (e.g., as a service). As a specific example application, application 120 can be a code development or management tool or part thereof that enables the creation, management, and / or delivery of code. Application 120 can be accessed via mobile applications, web applications, etc.

[0067] User device 110 may be a client device on the client side of operating environment 100, while security analysis manager 112 may be on the server side of operating environment 100. Security analysis manager 112 may include server-side software designed to work in conjunction with the client-side software on user device 110 to implement any combination of features and functionalities discussed in this disclosure. An example of such client-side software is application 120 on user device 110. This division of operating environment 100 is provided to illustrate an example of a suitable environment, and it should be noted that for each implementation, it is not necessary for any combination of user device 110 and security analysis manager 112 to remain separate entities.

[0068] In one embodiment, user equipment 110 and Figure 1 The illustrated security analysis manager 112 and data storage 114 are separate and distinct. In another embodiment, user device 110 is integrated with one or more of the illustrated components. For example, user device 110 may include the functionality described with respect to security analysis manager 112. For clarity, this document describes an embodiment in which user device 110, security analysis manager 112, and data storage 114 are separate, while understanding that this may not be the case in various conceivable configurations.

[0069] As described, user devices such as user device 110 can facilitate automated code security analysis. Specifically, user device 110 can facilitate security analysis manager 112 to obtain code security data and, in response, provide a view of indications or representations of security risks identified based on the security analysis of the code. As described herein, user device 110 can be operated by an individual or set of individuals who wish to initiate code security analysis and / or view the associated results. In some cases, user device 110 can be operated by code developers or administrators. Alternatively or additionally, user device 110 can be operated by an individual (e.g., a recipient of a software product) who wishes to analyze any security issues associated with the code.

[0070] In some cases, code security analysis can be initiated at user device 110. In this regard, the user can provide or select code security data for analysis to identify any security risks. Code security data generally refers to any data that can be used to perform code security analysis. Code security data can include, for example, design data, code data, threat data, security requirement data, PS data, and / or OS data.

[0071] For example, users such as software product developers can input, provide, or select various types of code security data for performing code security analysis. For instance, users can input or select design data, code data, threat data, security requirement data, PS data, and / or OS data via a user interface. In some cases, users can navigate to and select the relevant code security data they wish to use for performing code security analysis and choose to upload the code security data (e.g., via user device or data storage 114). As another example, users can select one or more code security data based on a list of candidate code security data displayed via user device (e.g., a list of different product designs may be presented for selection). Code security data can be in any of several formats, which may vary depending on the type of code security data.

[0072] The input or selection of code security data can be provided via an application 120 operating on user device 110. In this regard, user device 110 can allow users to input, select, or otherwise provide code security data, such as design data, code data, threat data, security requirement data, PS data, and / or OS data, via application 120. Application 120 can facilitate the input of code security data in verbal, text input, document, image, or other formats. Such code security data can be input at user device 110 in any way. For example, when accessing a specific application (e.g., a code development and / or management application), input tools can be presented to the user, or the user can navigate to them, to input or select various code security data desired for performing code security analysis.

[0073] Based on the code security analysis performed, representations or security risk data of one or more potential security risks identified by the code security analysis can be presented to a user via an application 120 operating on user device 110. In this way, any security risks identified through code security analysis can be displayed to individuals or entities wishing to view potential code security risks. In some cases, application 120 may enable the user to modify code (or other code security data) to resolve one or more identified code security risks. Alternatively or additionally, code can be automatically modified to resolve one or more identified code security risks.

[0074] User equipment 110 can communicate with security analysis manager 112 to provide code security data or its indications and / or obtain representations of one or more potential code security risks. In various embodiments, for example, a user may utilize user equipment 110 to provide code security data via network 122. For example, in some embodiments, network 122 may be the Internet, and user equipment 110 interacts with security analysis manager 112 to provide code security data for performing code security analysis. In other embodiments, for example, network 122 may be a corporate network associated with an organization. It will be apparent to those skilled in the art that any number of other implementation scenarios are also possible.

[0075] Continue to refer to Figure 1The security analysis manager 112 can be implemented as a server system, program module, virtual machine, component of one or more servers, network, etc. At a high level, the security analysis manager 112 manages the analysis of code to identify any potential security risks associated with it. Specifically, security analyses associated with a software product or parts thereof can be performed to identify any potential security risks associated with the software product. To perform code security analysis, various types of code security data can be analyzed to perform a comprehensive analysis. For example, design data, code data, threat data, security requirement data, PS data, and / or OS data can be analyzed to identify potential security risks associated with the software product or parts thereof. In this regard, the various embodiments described herein can perform design analysis, design-to-code analysis, and / or code analysis to identify potential security risks associated with various types of code security data. Design analysis analyzes the design associated with the product regarding various types of data (such as threat data, security requirement data, PS data, and / or OS data). In this way, design analysis identifies security deficiencies or defects associated with the design of the software product that may lead to security risks. Design-to-code analysis typically analyzes the implementation or coding associated with the design of the product. For example, design-to-code analysis can identify whether a design is implemented as intended. Where the design is not implemented as intended via code, potential security risks can be exposed, thus enabling their identification. Code analysis typically analyzes programming or implementation issues in the code that may lead to security risks. In this regard, even if a product can be implemented as intended via design, that implementation may still lead to security risks (e.g., via integer overflow, memory corruption, etc.). Utilizing various code analysis techniques enables robust and comprehensive security analysis of products from design to implementation.

[0076] Furthermore, the embodiments described herein perform code security analysis in an efficient manner. For example, AI techniques such as one or more LLMs are used to facilitate efficient and comprehensive analysis. Specifically, such techniques can be used to identify relevant data to be analyzed and to perform security analysis on that data.

[0077] In operation, in one example, various code security data are obtained at Security Analysis Manager 112. For example, design data, code data, threat data, security requirement data, PS data, and / or OS data can be obtained at Security Analysis Manager 112 via user devices (such as user device 110) and / or data sources (such as data storage 114). Security Analysis Manager 112 can then perform various analyses associated with the code security data to identify any potential security risks associated with a particular software product or portions thereof. In various embodiments, Security Analysis Manager 212 can perform design analysis, design-to-code analysis, and / or code analysis associated with the software product to identify any potential security risks. For performing design analysis, design data can be compared with threat data, security requirement data, PS data, and / or OS data to assess whether the design has any design flaws that could indicate potential security risks. For design-to-code analysis, design data can be compared with code data to assess whether the desired design is captured in the implementation. Potential security risks can be identified where inconsistencies exist between the design and code. For code analysis, the code is analyzed to assess data flows and / or patterns that indicate potential security risks. It can provide representations or security risk data of potential security risks associated with the software product and identified through security analysis, for display to the user, for example, via a user device such as user device 110.

[0078] Now go to Figure 2 , Figure 2 The illustration depicts an example implementation for facilitating the management of automated security analytics via a security analytics manager 212. The security analytics manager 212 can communicate with a data storage 214. The data storage 214 is configured to store various types of information accessible to the security analytics manager 212 or other servers or components. In various embodiments, the security analytics manager 212 and one or more user devices (such as…) Figure 1 User equipment 110 can provide data to data storage 214 for storage, and this data can be retrieved or referenced by any such component. Thus, data storage 214 can store design data, code data, threat data, security requirement data, PS data, OS data, security risk data, or combinations thereof, or representations thereof.

[0079] In operation, the security analysis manager 212 is typically configured to manage the analysis of code security. Specifically, the security analysis manager 212 manages the analysis of various security code data to identify any potential security risks associated with the code of a software product or parts thereof. In various embodiments, the security analysis manager 212 includes a code security data manager 220, a design analyzer 230, a design-to-code analyzer 240, a code analyzer 250, a results provider 260, and a solution manager 270. According to the embodiments described herein, the security analysis manager 212 may include any number of other components not shown. In some embodiments, one or more of the illustrated components 220, 230, 240, 250, 260, and 270 may be integrated into a single component or may be divided into several distinct components. Components 220, 230, 240, 250, 260, and 270 may be implemented on any number of machines and may be integrated with any number of other functions or services as needed.

[0080] The Code Security Data Manager 220 is typically configured to manage code security data that can be used to perform code security analysis. Code security data used for code security analysis can include various types of data. As an example only, code security data may include design data, code data, threat data, security requirement data, PS data, and / or OS data.

[0081] To manage code security data, code security data manager 220 may include code security data acquirer 222 and data preprocessor 224. According to the embodiments described herein, code security data manager 220 may include any number of other components not shown. In some embodiments, one or more of the illustrated components 222 and 224 may be integrated into a single component or may be divided into several distinct components. Components 222 and 224 may be implemented on any number of machines and may be integrated with any number of other functions or services as needed.

[0082] Code security data acquirer 222 is typically configured to acquire code security data. For example, code security data 282 can be acquired as input data 280 to security analysis manager 212. In some cases, code security data or portions thereof can be acquired via a user device. In this manner, code security data can be transmitted from the user device to security analysis manager 212 via a network. For example, based on the user's selection (or instruction) of code security data, corresponding code security data can be transmitted from the user device to security analysis manager 212. For example, a user, such as a code developer or administrator, can select or input a set of design documents associated with the code to be analyzed, a set of code to be analyzed, etc., via a user computer. Alternatively or additionally, code security data or portions thereof can be acquired via data storage (such as data storage 214) or other data sources. For example, based on the user's selection of a specific design document associated with the code, code security data acquirer 220 can retrieve or access the corresponding design document from data storage 214. As another example, based on a specific product to be analyzed, a specific PS dataset and / or OS dataset can be accessed or retrieved from data storage 214.

[0083] The code security data acquirer 222 can acquire any type and / or quantity of code security data. In this regard, the code security data acquired for performing code security analysis can include a wide variety of data types. By way of example only, code security data can include design data, code data, threat data, security requirement data, PS data, and / or OS data. Such data can be in any format suitable for capturing any amount of it.

[0084] Design data typically refers to data that indicates the design associated with code and / or a software product. In this context, design data refers to information that outlines or indicates the intended design and structure of a software product. Design data can include various forms of documentation describing how the software should be organized, how it should function, and / or how different components interact with each other. In this way, the various components of design data can include design documents, architecture descriptions, and specifications. Design documents can include detailed documents specifying the desired architecture and design of the software. These documents provide developers with a blueprint to follow during the implementation phase. Architecture descriptions can include high-level descriptions of the software architecture, including the system's structure, relationships between components, and overall design principles. Specifications can include detailed descriptions of specific aspects or elements of the software product, which may include functional specifications, interface descriptions, and data flow diagrams. Design data can be represented in various formats, including, for example, text documents, diagrams and models, spreadsheets, code comments, presentation slides, web pages, design tools, etc.

[0085] Code data typically refers to the code to be analyzed. Code data can refer to the actual source code written for a software product, which may include instructions and statements written in a programming language that define the functionality and behavior of the software product. Various components of code data may include source code files (e.g., a main file containing human-readable instructions written in a programming language), configuration files (e.g., files defining settings and parameters for the software's execution environment, build process, or runtime behavior), script files (e.g., scripts written to automate tasks related to the development, testing, deployment, or maintenance of the software), library and dependency files (e.g., external code libraries and dependencies on which the software depends), and / or test code (e.g., code written for testing purposes). Code data can be represented in various formats, such as source code files (e.g., plain text files, configuration files, script files, library files, etc.).

[0086] Threat data generally refers to data that indicates security threats. Threat data can include information identifying and / or describing potential or actual security threats to an organization, software product, and / or the broader community. In this respect, threat data is highly valuable for performing security analysis. Various components of threat data can include threat lists (e.g., an enumeration of identified threats specific to an organization, software product, and / or the broader community), threat descriptions (e.g., a detailed explanation of each threat, such as its nature, potential impact, etc.), incident reports (e.g., documentation of security breaches, such as how vulnerabilities were discovered, exploited, and their consequences), vulnerability data (e.g., information about specific weaknesses in a system or software that could be exploited by threats), threat intelligence (e.g., data collected from various sources providing insights into emerging threats and trends), and / or mitigation strategies (e.g., recommendations and actions for addressing and mitigating identified threats). As described, in some cases, threat data can include threat lists, such as threats identified within an organization (e.g., an organization developing a software product for which security analysis is to be performed). Additionally or alternatively, threat data can include threats identified within the broader community or other third-party organizations or entities. For example, after a security breach has occurred, an organization or group can identify threats, or identify threats based on potential future security breaches. In some cases, threat data may be product-specific. In others, threat data may include all identified threats. Threat data can be represented in various formats, such as text documents, spreadsheets, databases, JSON files, XML files, Security Information and Incident Management (SIEM) logs, threat intelligence feeds, incident response reports, etc.

[0087] Security requirements data typically refers to any requirements or guidelines generated to provide software security. In this way, security requirements are usually created to ensure the security of software products. This data can be an integral part of the development process and facilitates the establishment of necessary security measures from the design phase to deployment and maintenance. The various components of security requirements data can include, for example, requirement specifications (e.g., detailed descriptions of the security requirements of a software product or subsystem that need to be implemented to ensure security), guidelines (e.g., best practices and recommendations for implementing security measures during software development), documentation (e.g., comprehensive documents describing the security requirements of different components or subsystems that are typically aligned with the overall security architecture), access control requirements (e.g., specifications related to user roles and permissions), and compliance requirements (e.g., derived from industry standards, regulations, and legal mandates). As an example, from a security perspective, different users can use a software product, such as administrators and users. Thus, security requirements can be used to ensure that users should not be administrators by default. Security requirements can be generated at any time. In some cases, security requirements are generated by the product design team during the design phase when designing the product or subsystem. Furthermore, security requirements data can be provided in any number of formats, such as text documents, spreadsheets, specification documents, charts and models, compliance checklists, policy documents, JSON and XML files, requirements management tools, etc.

[0088] As described, PS data typically refers to information and findings collected internally by an organization (e.g., via an offensive security review team). In this regard, PS data includes examples of security issues found in the code or design of various projects, which can then be used to improve the security of the entire organization (e.g., the company). The various components of PS data may include findings (e.g., a set of security issues identified in code or design that may include actual implementation problems and design flaws), examples (e.g., specific instances of bugs or vulnerabilities found within the company that are annotated and documented), analysis reports (e.g., detailed reports describing security issues, such as the context of the discovery, its potential impact, and the methods used to discover the security issues), mitigation strategies, and / or annotated code (e.g., code snippets or complete code examples annotated to highlight the exact location and nature of the security issues). PS data can be in any number of formats, such as text documents, spreadsheets, reports, databases, JSON and XML files, annotated code files, security tool outputs, internal wikis or knowledge bases, etc.

[0089] Open source data generally refers to any data collected from open source projects, specifically focusing on security issues. Therefore, open source data can serve as a library of lessons learned from other projects, which can be used to analyze and improve the security of an organization's own designs or code. Components of open source data can include security bugs (e.g., a collection of security vulnerabilities and issues identified in open source projects), case studies, examples (e.g., specific instances of security issues from open source libraries or projects archived for future analysis), analytical reports, comparative data, etc. Open source data can include security issues arising from developers' misunderstandings of the meaning of a design and thus its incorrect implementation. Open source data can be in any number of formats, such as text documents, spreadsheets, reports, databases, JSON and XML files, commented code files, security tool output, internal wikis or knowledge bases, etc.

[0090] Based on the acquired code security data, data preprocessor 224 is typically configured to preprocess the code security data. In various embodiments, data preprocessor 224 may convert or transform the acquired code security data, or a portion thereof, into a format suitable for analysis. Thus, in some embodiments, data preprocessor 224 converts the code security data, or a portion thereof, into vector data. In some cases, all code security data may be converted into vector data. In other cases, a portion of the code security data may be converted into vector data. For example, specific types of code security data may be converted into vector data. By way of example only, code security data from which a large language model can retrieve information may be converted into vector data. Vector data, or vectors, are typically used to represent data points in a high-dimensional space that can be used in machine learning, natural language processing, and other techniques involving embeddings or feature representations.

[0091] In some cases, code security data in text form can be indexed and transformed into vector data. For example, PPT, DOC, and PDF files (e.g., design documents, security requirements, etc.) can be indexed and converted to a vector storage format using LangChain or a similar framework. LangChain is a framework designed to facilitate the development of applications that leverage language models in conjunction with other data sources and tools. LangChain allows for the creation of operation chains that can include language model hints, data retrieval, and other processing steps. In other cases, code security data in image form (e.g., threat data) can be translated into text and then transformed into vector data. For example, an image can be translated into text using an image-to-text model.

[0092] In various embodiments, vector data may be stored in a data store such as vector storage 212. In some cases, the data store may be a vector store that is typically designed to manage high-dimensional vector data. In some cases, vector data and non-vector data may be stored separately. Vector storage is merely one example for storing data, and the embodiments are not limited to this document.

[0093] The code security data manager 220 can acquire and / or preprocess data at any time. For example, in some cases, the code security data manager 220 can acquire code security data based on the expiration of a duration, process this data, and store it for later use. In other cases, the code security data manager 220 can acquire code security data based on the occurrence of an event, then store that data for later use, and / or perform security analysis on this data. For example, various security data can be acquired or processed based on user choices regarding security associated with the code. As another example, data can be acquired and processed via the code security data manager 220 based on user-provided code security data.

[0094] Design analyzer 230 is typically configured to analyze designs associated with code. In this way, design analyzer 230 can evaluate software product designs created by architects, focusing on the conceptual and structural aspects of the design rather than its implementation. Design analysis may include examining the intended functionality and interactions of various components (e.g., within a system) to ensure that the design accurately represents how the system should operate and / or how different elements communicate with each other without introducing security risks. Thus, design analyzer 230 facilitates the identification of potential security risks during the design phase, independent of any coding or implementation work.

[0095] Design Analyzer 230 can analyze product designs or portions thereof associated with various types of code security data, such as threat data, security requirement data, PS data, and / or OS data. In this way, Design Analyzer 230 can efficiently and effectively analyze product designs associated with various types of code security data to identify potential security issues.

[0096] The design of a product associated with a threat can be analyzed to identify any flaws in the design that relate to the identified threat (e.g., existing or anticipated threats). To analyze one or more designs associated with threat data, design analyzer 230 can analyze one or more designs associated with various threats to identify any potential security risks associated with the product design. Potential security risks can be identified where the product design does not include any design data indicating mitigation of the corresponding threat. As an example, suppose the identified threat relates to any individual capable of entering and starting the vehicle. In this regard, the product design or portions thereof are analyzed to identify whether there is a mechanism in the design that locks the vehicle to mitigate the threat, allowing only the owner to open and start the vehicle.

[0097] As described, threat data can include a list of threats (e.g., identified by an organization, a broader group, etc.). Threats can include actual or potential threats, that is, threats that have been detected based on previous implementations or identified as potential threats based on analysis of the data. The design of a software product, identified via design data, can be analyzed by design analyzer 230 on a per-threat basis to identify whether the design mitigates that threat. This process can be iterated for various threats. Any number of threats associated with one or more product designs can be analyzed. For example, in some cases, threats for which security problem analysis is to be performed can be identified as product-related or relevant threats. In other cases, security risk analysis can be performed on any threats obtained.

[0098] Design analyzer 230 can obtain threat data to be analyzed based on design data in any number of ways and at any time. For example, a threat list can be obtained as input data 250, obtained via code security data manager 220, accessed via data storage 214, etc. Furthermore, threat data can be obtained based on the expiration of a duration or the occurrence of an event for analysis by design analyzer 230. For example, threats can be obtained and analyzed by performing security analysis or design analysis related to a software product based on user instructions.

[0099] In operation, where the design analyzer 230 obtains a threat dataset, it can analyze the design data associated with each threat. To analyze the design data associated with threats, the design analyzer 230 can identify relevant design data corresponding to the threat. As described, in one example, the design data is represented via vector data stored in a data storage (e.g., vector storage). In this way, for a threat, the design analyzer 230 can identify relevant design data based on a search of the corresponding vector data. In other words, the design analyzer 230 can identify design fragments or portions in design literature that match or correspond to a specific threat. Based on the identified matching design literature fragments, the design analyzer 230 can analyze whether the product design addresses the corresponding security threat. If the design does not address or mitigate the corresponding security threat, the threat is identified as a potential security risk. In some implementations, if there is any uncertainty regarding whether the design mitigates the threat, the potential security risk can be entered and identified as such. This process can be performed for various threats (e.g., each threat in a threat set): (e.g., via vector storage) identifying relevant or matching design data associated with the threat, and then analyzing this design data.

[0100] In various embodiments, AI techniques, such as machine learning or other techniques, are used to identify and / or analyze relevant parts of the design associated with the threat. In this way, design analyzer 230 may include or use AI to identify and / or analyze relevant parts of the design. Design analyzer 230 may include or access any number of AI models or techniques. As an example, a machine learning model in the form of an LLM may be used to identify and / or analyze relevant parts of the design associated with the threat. A language model is a statistical and probabilistic tool (e.g., via next sentence prediction [NSP] or masked language model [MLM]) that determines the probability of a given sequence of words appearing in a sentence. In short, it is a tool trained to predict the next word in a sentence. When a language model is trained on a large amount of data, it is called a large language model (LLM). Specifically, an LLM refers to a language model that includes a neural network with a large number of parameters, trained using self-supervised learning on a large amount of unlabeled text. Typically, an LLM has billions or more parameters. Some examples of LLMs are Google's BERT and OpenAI's GPT-2, GPT-3, GPT-4, and GPT-4o, and / or NVIDIA's NVLM 1.0 and future versions. For example, GPT-3 is a large language model with 175 billion parameters trained on 570GB of text. The capabilities of these models range from writing simple articles to generating complex computer code (all of which are limited to unsupervised learning). Thus, LLMs are very large deep neural networks (billions to hundreds of billions of parameters) and are trained on massive amounts of text to understand, process, and generate human natural language. While some examples provided herein include single-mode generative models, other models, such as multimodal generative models, should also be considered within the scope of the embodiments described herein. Typically, multimodal models are generated to make predictions based on different types of modalities (e.g., text and images). In some embodiments, the design analyzer 230 takes the form of an LLM or uses an LLM, but various other artificial intelligence models or techniques may be used additionally or alternatively. Other models or techniques may be used in this paper, including but not limited to small language models.

[0101] To utilize the LLM, the design analyzer 230 can facilitate the generation of one or more prompts for input into the LLM to identify and / or analyze relevant design data associated with threat data. Any number of prompts can be generated to identify and / or analyze relevant design data associated with threat data. As an example, a first prompt can be generated, and this first prompt can be used to identify relevant design data associated with one or more threats; and a second prompt can be generated, and this second prompt can be used to analyze the relevant design data associated with one or more threats (e.g., for security risk analysis).

[0102] In this manner, the design analyzer 230 can generate design-threat identification cues used to identify design data associated with one or more threats. In this regard, the design-threat identification cues may include instructions for identifying design data that is related to or matches a threat. In some cases, design-threat identification cues can be generated for a specific threat. In other cases, design-threat identification cues can be generated for a set of threats. Design-threat identification cues may include one or more threats as context. In this manner, the design-threat identification cues can be used to search a data store containing representations of design data to identify design data associated with one or more threats. Figure 3A An example hint is provided. Figure 3A In this context, input threat 301 is a variable representing a threat. In this way, during program execution, the input threat can be populated with threats (e.g., threat descriptions). Design data (e.g., design literature) can be retrieved from a database. Figure 3A As shown, the prompt includes instructions 302 related to analyzing input threats, instructions 303 related to identifying matching design components, target output 304, and example output 305.

[0103] In practice, as an example, a design-threat identification cue, including the threat as context, can be converted into a vector (e.g., an embedding) using, for example, an LLM or embedding model, representing the semantic meaning of the cue in a high-dimensional space. The LLM can then use the generated vector to query a data store containing pre-computed vectors or embeddings of design data. The search performed may result in, for example, identifying or retrieving the vector most similar to the cue's vector based on a similarity metric such as cosine similarity. Such search results can include data entries (e.g., portions of design documents) associated with the closest vector identified as semantically most relevant to the design-threat identification cue. In this way, design data related to the threat and / or its mitigation can be automatically identified based on the obtained design data.

[0104] Based on identified design data related to or relevant to threats, design analyzer 230 can facilitate the analysis of such identified design data associated with threats. In some cases, LLM or other AI technologies can be used to perform this type of analysis. In this regard, where matching design data fragments are identified, the design data can be analyzed to assess whether and / or how the product design addresses or mitigates one or more corresponding security threats. In this way, design analyzer 230 can generate design-threat analysis tips used to analyze design data related to one or more threats. In some cases, design-threat analysis tips can be generated for a specific threat. In other cases, design-threat analysis tips can be generated for a set of threats. Design-threat analysis tips may include instructions for analyzing design data associated with threats (e.g., identifying whether the design data mitigates the threat). In some cases, design-threat analysis tips may include one or more threats as context and / or indications of relevant design data as context. For example, design-threat analysis tips may include identified relevant design data or references or indications including relevant design data for performing analysis. In this way, the analysis of design data used to analyze threats is minimized to design data that was previously identified as relevant, so as to perform more efficient analysis.

[0105] Figure 3B An example hint is provided. Figure 3B In this context, input threat 308 and design document fragment 309 can be variables representing threats and design data, respectively. In this way, input threat 308 can be populated with threats (e.g., threat descriptions), and design document fragment 309 can be input along with the identified design data. Figure 3B As shown, the prompt includes instructions 310 related to analyzing input threats, instructions 311 related to reviewing design document fragments, instructions 312 related to evaluating design fragments, instructions 313 related to identified problems, instructions 314 related to marking design fragments, target output 315, and example output 316.

[0106] Design-threat analysis tips can be provided as input to an LLM to perform analysis of design data associated with threats. Specifically, design data can be analyzed to identify whether the design data mitigates potential security threats. An LLM can provide security risk data as output, which includes any data associated with security risks. This type of security risk data can include various types of information. As an example, it can provide an indication of whether a specific threat has been mitigated. As another example, it can provide an indication that design data is being analyzed. As yet another example, it can provide the degree to which a threat is likely to be mitigated. As yet another example, it can provide the degree of risk of a threat (e.g., if not properly mitigated). Security risk data can be provided at any level of granularity. For example, it can provide a general indication of a design flaw. As another example, it can provide an explanation of why a design flaw exists.

[0107] Although described as a two-step prompt generation and LLM analysis process (e.g., identifying relevant data and then using that data for security analysis), the implementation can include any number of steps (e.g., a single prompt generation and analysis step). Furthermore, while the individual examples perform this two-step process for each threat (e.g., threats in the obtained threat set), it is also possible to perform this process for a threat set or a batch of threats.

[0108] Design analyzer 230 can also analyze the design of a product associated with safety requirements to identify whether the design has any defects related to the identified safety requirements. To analyze one or more designs associated with safety requirement data, design analyzer 230 can analyze one or more designs associated with various safety requirements to identify one or more potential safety risks associated with the product design. Potential safety risks can be identified where the design does not include any indication of implementing or meeting the corresponding safety requirements.

[0109] As described, security requirement data may include any indication of security requirements or a security checklist for the software product. The design of the software product identified via design data can be analyzed by design analyzer 230 according to each security requirement to identify whether the security requirements are met in the design. This process can be iterated for various security requirements. Any number of security requirements associated with one or more product designs can be analyzed. For example, in some cases, security requirements for which security issue analysis is to be performed can be identified as product-related or relevant security requirements. In other cases, security issue analysis can be performed on any security requirements obtained.

[0110] Design analyzer 230 can obtain security requirement data to be analyzed based on design data in any number of ways and at any time. For example, a list of security requirements can be obtained as input data 280, obtained via code security data manager 220, accessed via data storage 214, etc. Furthermore, security requirement data can be obtained based on the expiration of a duration or the occurrence of an event for analysis by design analyzer 230. For example, security requirements can be obtained and analyzed by performing security analysis, design analysis, and / or design analysis associated with security requirement data based on user instructions.

[0111] In operation, where the design analyzer 230 obtains a dataset of safety requirements, it can analyze the design data associated with each safety requirement. To analyze the safety requirement data associated with each safety requirement, the design analyzer 230 can identify relevant design data corresponding to the safety requirement. As described, in one example, the design data can be represented via vector data stored in a data storage (e.g., vector storage). In this way, for a safety requirement, the design analyzer 230 can identify relevant design data based on a search of the corresponding vector data. In other words, the design analyzer 230 can identify fragments in design documentation that match or correspond to a specific safety requirement. Based on the identified matching design documentation fragments, the design analyzer 230 can analyze whether the product design addresses the corresponding safety requirement. If the design does not address or meet the corresponding safety requirement, then that safety requirement is identified as a potential safety risk. In some implementations, if there is any uncertainty regarding whether the design has met the safety requirement, a potential safety risk can be entered and identified as such. This process can be performed for various security requirements (e.g., each security requirement in a set of security requirements): (e.g., via vector storage) identify relevant or matching design data associated with the security requirements, and then analyze that design data.

[0112] In various embodiments, AI technologies, such as machine learning or other techniques, are used to identify and / or analyze relevant parts of the design associated with security requirements. In this way, design analyzer 230 may include or use AI to identify and / or analyze relevant parts of the design. Design analyzer 230 may include or access any number of AI models or techniques. As an example, a machine learning model of the form LLM is used to identify and / or analyze relevant parts of the design associated with security requirements, as described herein.

[0113] To use the LLM, the design analyzer 230 can facilitate the generation of one or more prompts for input into the LLM to identify and / or analyze relevant design data associated with the safety requirement data. Any number of prompts can be generated to identify and / or analyze relevant design data associated with the safety requirement data. As an example, a first prompt can be generated, and this first prompt can be used to identify relevant design data associated with one or more safety requirements; and a second prompt can be generated, and this second prompt can be used to analyze the relevant design data associated with one or more safety requirements.

[0114] In this manner, the design analyzer 230 can generate a design-safety requirement identification prompt, which is used to identify design data associated with one or more safety requirements. In some cases, a design-safety requirement identification prompt can be generated for a specific safety requirement. In other cases, a design-safety requirement identification prompt can be generated for a set of safety requirements. The design-safety requirement identification prompt may include instructions for identifying design data associated with or matching one or more safety requirements. In various embodiments, the design-safety requirement identification prompt may include one or more safety requirements as context. In this manner, the design-safety requirement identification prompt can be used to search a data store including representations of design data to identify design data associated with one or more safety requirements.

[0115] Figure 3C An example hint is provided. Figure 3C In this context, input requirement 318 is a variable representing a security requirement. In this way, during program execution, the input requirement can be populated with security requirements (e.g., threat descriptions). Design data (e.g., design literature) can be retrieved from a database. Figure 3C As shown, the prompts include instructions 319 related to the analysis input requirements, instructions 320 related to the identification and matching design section, target output 321, and example output 322.

[0116] In practice, as an example, a design-safety requirement identification cue, which includes one or more safety requirements as context, can be converted into a vector (e.g., an embedding) using, for example, an LLM or embedding model, representing the semantic meaning of the cue in a high-dimensional space. The LLM can then use the generated vectors to query a data store containing pre-computed vectors or embeddings of design data. The performed search may result in identifying or retrieving the vector most similar to the cue's vector, for example, based on a similarity metric such as cosine similarity. Such search results can include data entries (e.g., portions of design documents) associated with the closest vector identified as semantically most relevant to the design-safety requirement cue. In this way, design data related to safety requirements and / or their attainment can be automatically identified based on the obtained design data.

[0117] Based on the identified design data related to or relating to safety requirements, design analyzer 230 can facilitate the analysis of such identified design data associated with safety requirements. In some cases, LLM or other AI technologies can be used to perform such analyses. In this regard, where matching design data fragments are identified, the design data can be analyzed to assess whether and / or how the product design addresses one or more corresponding safety requirements. In this way, design analyzer 230 can generate design-safety requirement analysis prompts used to analyze design data related to one or more safety requirements. In some cases, design-safety requirement analysis prompts can be generated for a specific safety requirement. In other cases, design-safety requirement analysis prompts can be generated for a set of safety requirements. Design-safety requirement analysis prompts may include instructions to analyze design data according to safety requirements to identify whether the safety requirement is addressed in the design data. In various embodiments, design-safety requirement analysis prompts may include indications of one or more requirements as context and / or related design data as context. For example, design-safety requirement analysis prompts may include identified related design data or references or indications including related design data for performing analysis. In this way, the analysis of product data used to analyze safety requirements is minimized to design data that was previously identified as relevant, so as to perform more efficient analysis.

[0118] Figure 3D An example hint is provided. Figure 3D In this context, input requirement 324 and design document fragment 325 can be variables. In this way, during program execution, the input requirement and design document fragment can be accessed via... Figure 3C The security requirements and design data for prompt recognition provided in the documentation are used to populate the data. For example... Figure 3DAs shown, the prompt includes instructions 326 related to analysis input requirements, instructions 327 related to reviewing design document fragments, instructions 328 related to evaluating design fragments, instructions 329 related to identifying problems, instructions 330 related to marking design fragments, target output 331, and example output 332.

[0119] Design-safety requirement analysis prompts can be provided as input to an LLM to perform analysis of design data associated with safety requirements. Specifically, design data can be analyzed to identify whether the design data meets safety requirements. In response, the LLM can provide safety risk data indicating or representing any data associated with any safety risk. Safety risk data can include various types of information. As an example, it can provide an indication of whether a specific safety requirement has been met. As another example, it can provide an indication of the design data being analyzed. As yet another example, it can provide the degree of likelihood of meeting safety requirements. As yet another example, it can provide the degree of risk of not meeting safety requirements (e.g., if not properly addressed). Safety risk data can be provided at any level of granularity. For example, it can provide a general indication of a design flaw. As another example, it can provide an explanation of why a design flaw exists.

[0120] As an example, suppose 28 design flaws are identified as potential security risks associated with the product design being analyzed. In addition to providing indications of the 28 design flaws, the severity level of the unmet security requirements can also be provided. For example, each design flaw could include an indication of low severity, medium severity, or high severity, indicating the degree of security risk that may arise from the failure to meet the corresponding security requirement. In other examples, numerical scores (e.g., between 1 and 10) or any other indicator can be used to visually reflect the severity of the potential security risks. For example, a Common Vulnerability Scoring System (CVSS) score can be used to reflect severity. In some embodiments, the LLM can determine or calculate the CVSS score based on the design flaws. Additionally or alternatively, statistical data associated with the design flaws can be identified. For example, the proportion of the identified design flaws relative to the total number of security requirements analyzed can be identified. As another example, the proportion of high-severity, medium-severity, or low-severity design flaws can be identified (e.g., based on CVSS scores).

[0121] Although described as a two-step prompt generation and LLM analysis process (e.g., identifying relevant data and then using that data for security analysis), the implementation can include any number of steps (e.g., a single prompt generation and analysis step). Furthermore, while the examples perform this two-step process for each security requirement (e.g., security requirements in a obtained set of security requirements), it is also possible to perform this process for a set of security requirements or a batch of security requirements.

[0122] Moving to design analysis associated with PS and / or OS data, design analyzer 230 can analyze one or more designs associated with various PS and / or OS data to identify potential security risks related to the product design. Potential security risks can be identified where the design data lacks any indication of mitigating security vulnerabilities corresponding to the PS and / or OS data.

[0123] As described, PS data typically refers to information and findings collected within an organization. In this regard, PS data can include examples of security issues found in the code or design of various projects, which can be used to improve the security of the entire organization (e.g., the company). OS data typically refers to data collected from open-source projects, specifically focusing on security issues. Thus, open-source data can serve as a library of lessons learned from other projects, which can be used to analyze and improve the security of the organization's own designs or code. PS and / or OS data can include questions from product design evaluations, vulnerability patterns based on previous reviews (e.g., examples of design flaws), annotated code snippets indicating flaws, etc.

[0124] Design analyzer 230 can analyze the design of a software product, such as those identified via design data, based on each PS pattern and / or OS pattern to identify whether the occurrence of the same or similar patterns is mitigated in the design. This process can be iterated for various PS patterns and / or OS patterns. Any number of PS patterns and / or OS patterns associated with one or more product designs can be analyzed. For example, in some cases, PS patterns and / or OS patterns for security issue analysis can be identified as product-related or relevant PS patterns and / or OS patterns. In other cases, security issue analysis can be performed on any PS patterns and / or OS patterns obtained. A PS pattern typically refers to an instance or pattern associated with a threat identified in the PS data. Similarly, an OS pattern typically refers to an instance or pattern associated with a threat identified in the OS data. For example, in some cases, a PS pattern may include data showing a specific threat or security issue identified in the PS data.

[0125] Design analyzer 230 can acquire PS data and / or OS data for design data analysis in any number of ways and at any time. For example, a list of PS and / or OS modes can be acquired as input data 280, obtained via code security data manager 220, accessed via data storage 214, etc. Furthermore, PS data and / or OS data can be acquired based on the expiration of a duration or the occurrence of an event for analysis by design analyzer 230. For example, PS data can be acquired and analyzed based on user instructions to perform security analysis, design analysis, and / or design analysis associated with PS data.

[0126] In operation, where the design analyzer 230 obtains PS and / or OS datasets, it can analyze the design data associated with each PS and / or OS pattern. To analyze the design data associated with PS and / or OS patterns, the design analyzer 230 can identify relevant design data corresponding to the PS and / or OS patterns. As described, in one example, the design data can be represented via vector data stored in a data storage (e.g., vector storage). In this way, for PS and / or OS patterns, the design analyzer 230 can identify relevant design data based on a search of the corresponding vector data. In other words, the design analyzer 230 can identify fragments in design literature that match a specific PS and / or OS pattern. Based on the identified matching design literature fragments, the design analyzer 230 can analyze whether the product design addresses the corresponding PS and / or OS patterns. Where the design fails to address, mitigate, or achieve the corresponding PS and / or OS patterns, the PS and / or OS patterns are identified as potential security risks. In some implementations, if any uncertainty exists regarding security risks, potential security risks can be entered and identified as such. This process can be performed for various PS modes and / or OS modes (e.g., each PS mode in a set of PS modes): (e.g., via vector storage) relevant or matching design data associated with the PS modes and / or OS modes can be identified, and then this design data can be analyzed.

[0127] As an example, design analysis using PS and OS data can include examining past issues to ensure they are not repeated in new designs. This process requires comparing portions of the current design with previously identified problematic code or design fragments, such as those indicated in PS and / or OS patterns. For instance, assuming the PS data includes design flaws, relevant code fragments can be isolated and included as PS patterns (e.g., annotated code fragments indicating the problem and its cause). The LLM can then compare this fragment indicated in the PS pattern with design fragments of the product currently being analyzed to identify similar issues. Thus, after identifying relevant portions of the design, the LLM can compare and / or contrast the identified relevant portions with PS patterns. By incorporating previous findings identified via PS and / or OS patterns, including annotated code fragments and their descriptions, the LLM can translate this knowledge into an analysis of the new design, thereby ensuring proactive resolution of potential security risks identified in past projects within the current project.

[0128] In various embodiments, AI techniques, such as machine learning or other techniques, can be used to identify and / or analyze relevant parts of the design associated with PS and / or OS modes. In this way, design analyzer 230 may include or use AI to identify and / or analyze relevant parts of the design. Design analyzer 230 may include or access any number of AI models or techniques. As an example, a machine learning model of the form LLM is used to identify and / or analyze relevant parts of the design associated with PS and / or OS modes, as described herein.

[0129] To use the LLM, the design analyzer 230 can facilitate the generation of one or more prompts for input into the LLM to identify and / or analyze relevant design data associated with PS data and / or OS data. Any number of prompts can be generated to identify and / or analyze relevant design data associated with PS data and / or OS data. As an example, a first prompt can be generated, and this first prompt can be used to identify relevant design data associated with one or more PS patterns and / or OS patterns; and a second prompt can be generated, and this second prompt can be used to analyze relevant design data associated with one or more PS patterns and / or OS patterns.

[0130] In this manner, the design analyzer 230 can generate design-PS identification hints or design-OS identification hints, which are used to identify design data associated with PS data or OS data, respectively. In some cases, design-PS identification hints or design-OS identification hints can be generated for specific PS patterns and / or OS patterns. In other cases, hints can be generated for a set of PS and / or OS patterns. A design-PS identification hint can include one or more PS patterns as context. In this manner, the design-PS identification hint can be used to search a data store containing representations of design data to identify design data associated with one or more PS patterns. Similarly, a design-OS identification hint can include one or more OS patterns as context. In this manner, the design-OS identification hint can be used to search a data store containing representations of design data to identify design data associated with one or more OS patterns.

[0131] Figure 3E An example hint is provided. Figure 3E In this context, input design defect pattern 334 is a variable representing (e.g., identified from one or more previous reviews associated with OS data and / or PS data) a design defect. Design data (e.g., design literature) can be retrieved from a database. Figure 3E As shown, the prompt may include instructions 335 related to analyzing input design defect patterns, instructions 336 related to identifying matching design parts, instructions 337 related to evaluating design parts, instructions 338 related to identifying problems, instructions 339 related to marking design parts, target output 340, and example output 341.

[0132] In practice, as an example, a design-PS identification cue, which includes one or more PS patterns as context, can be converted into a vector (e.g., an embedding) using, for example, an LLM or embedding model, representing the semantic meaning of the cue in a high-dimensional space. The LLM can then use the generated vectors to query a data store containing pre-computed vectors or embeddings of design data. The performed search can result in identifying or retrieving the vector most similar to the cue's vector, for example, based on a similarity metric such as cosine similarity. Such search results can include data entries (e.g., portions of a design document) associated with the closest vector of data identified as semantically most relevant to the design-PS identification cue. In this way, design data related to PS patterns and / or their mitigations can be automatically identified based on the obtained design data.

[0133] Based on the identified design data related to or relating to PS and / or OS modes, the design analyzer 230 can facilitate the analysis of such identified design data associated with PS or OS modes. In some cases, LLM or other AI techniques can be used to perform such analyses. In this regard, where matching design data fragments are identified, the design data can be analyzed to evaluate whether and / or how the product design addresses or mitigates one or more corresponding PS modes and / or one or more OS modes. In this way, the design analyzer 230 can generate design-PS analysis hints or design-OS analysis hints, which are used to analyze the design data associated with the PS or OS modes, respectively. In some cases, design-PS analysis hints and / or design-OS analysis hints can be generated separately for specific PS modes and / or OS modes. In other cases, design-PS analysis hints can be generated for a set of PS modes and / or for a set of OS modes. The design-PS analysis hints may include indications of one or more PS modes and / or related design data as context. For example, design-PS analysis prompts may include identified relevant design data or references or instructions that include relevant design data for performing the analysis. Similarly, design-OS analysis prompts may include one or more OS modes as context and / or instructions that include relevant design data as context. For example, design-OS analysis prompts may include identified relevant design data or references or instructions that include relevant design data for performing the analysis. In this way, the analysis of product data used to analyze PS data and / or OS data is minimized to the previously identified relevant design data, enabling more efficient analysis.

[0134] Design-PS analysis tips and / or design-OS analysis tips can be provided as input to the LLM to perform analysis on design data associated with PS data and / or OS data, respectively. Specifically, design data can be analyzed to identify whether the design data mitigates or resolves PS patterns and / or OS patterns. The LLM can provide security risk data as output. This type of security risk data can include various types of information. As an example, security risk data provided as output may include an indication of whether a particular PS pattern and / or OS pattern has been mitigated. As another example, it may provide an indication of the design data being analyzed. As yet another example, it may provide the degree to which PS patterns and / or OS patterns are likely mitigated. As yet another example, it may provide the degree to which the risk of PS patterns and / or OS patterns has not been mitigated (e.g., if not properly addressed). Security risk data can be provided at any level of granularity. For example, it may provide a general indication of a design flaw. As another example, it may provide an explanation of why a design flaw exists.

[0135] Although described as a two-step prompt generation and LLM analysis process (e.g., identifying relevant data and then using that data to perform security analysis), the implementation can include any number of steps (e.g., a single prompt generation and analysis step). Furthermore, while the individual examples perform this two-step process for each PS or OS pattern (e.g., PS or OS patterns in the obtained pattern set), this process can also be performed for a set of patterns or a batch of patterns.

[0136] In some cases, the design analyzer 230 can perform various analyses sequentially or concurrently. For example, in some cases, design data associated with threat data, security requirement data, and PS / OS data can be analyzed sequentially (in any order). In other cases, design data associated with threat data, security requirement data, and / or PS / OS data can be analyzed concurrently. Furthermore, depending on the implementation, it is not necessary to perform each analysis type. For example, where only threat data and security requirement data are available, security analysis can be performed only on the design associated with the threat data and security requirement data.

[0137] Design-to-code analyzer 240 is typically configured to perform design-to-code mapping analysis. In this respect, design-to-code analyzer 240 can map product designs to actual code implementations to identify whether developers have accurately interpreted and reflected the design in the code. At a high level, design-to-code analyzer 240 performs design-to-code mapping. Specifically, design data (e.g., design documentation) and corresponding code data (e.g., a code repository) are obtained. As an example, for various functions in a function set, a search can be performed (e.g., via query) to identify corresponding design fragments (e.g., in a data store such as a data storage). Thus, design data (such as how a door's function should work) is mapped to its corresponding actual code implementation. Using an LLM, the codebase and design documentation can be examined to find matching elements. The LLM is then prompted to search the source code stored in the data store to locate the specific code fragment that implements the function. The mapped data can then be analyzed to identify any inconsistencies that may be or indicate security issues. In this respect, once these mappings between design and code are established, inconsistencies in each mapping pair are further analyzed. LLM helps identify any deviations or errors in the code that may arise from misunderstandings of the design, thereby ensuring that the implementation is aligned with the original design intent.

[0138] In one embodiment, the design-to-code analyzer 240 may include or use artificial intelligence techniques such as machine learning or other techniques to map data and / or analyze the mapped data. Thus, the design-to-code analyzer 240 may include or access any number of AI models or techniques. As an example, a machine learning model in the form of an LLM is used to map designs to code and / or analyze such mappings. The same or different LLM or AI techniques may be used as described above.

[0139] To use the LLM, the design-to-code analyzer 240 can facilitate the generation of one or more hints for input into the LLM to map design data to code data associated with the product. Any number of hints can be generated to map and / or analyze the associated data. As an example, a first hint can be generated and used to map design data to code data; and a second hint can be generated and used to analyze the mapped data.

[0140] In this way, the design-to-code analyzer 240 can generate design-to-code mapping hints, which are used to map design data to code data associated with the product. In some cases, design-to-code mapping hints can be generated for specific design or code data. In other cases, design-to-code mapping hints can be generated for design features or sets of code features. Design-to-code mapping hints can include design data and / or code data as context. In this way, design-to-code mapping hints can be used to search a data store that includes representations of design data to identify design data associated with one or more code snippets (such as functions).

[0141] Figure 3F An example hint is provided. Figure 3F In this context, the input variables include design fragment 344 and source code repository 345, and the output includes one or more functions that implement the design fragment. For example... Figure 3F As shown, the prompt includes instructions 346 related to analyzing design fragments, instructions 347 related to identifying matching implementations in the source code, instructions 348 related to evaluating implementations, instructions 349 related to creating design-implementation mappings, instructions 350 related to marking states, target output 351, and example output 352.

[0142] In practice, as an example, a design-code mapping hint, which includes one or more code functions as context, can be converted into a vector (e.g., an embedding) using, for example, an LLM or embedding model, representing the semantic meaning of the hint in a high-dimensional space. The LLM can then use the generated vectors to query a data store containing pre-computed vectors or embeddings of design data. The search performed can result in identifying or retrieving the vector most similar to the hint's vector, for example, based on a similarity metric such as cosine similarity. Such search results can include data entries (e.g., portions of a design document) associated with the closest vector identified as semantically most relevant to the design-code mapping hint. In this way, design data related to code snippets (such as code functions) is automatically identified. Thus, for each function, a query or hint is used to identify the corresponding design data snippet in order to match the design with the implementation.

[0143] If no matching code snippets, such as code functions, or relevant design data are identified, non-matches can be identified and recorded. In this way, a non-match record means that the code snippet does not match the product design.

[0144] Based on design data identified as related to or associated with code snippets (e.g., code functions), the code-to-design analyzer 230 can generate mapping tables that include various design-code mappings. For example, it can generate mappings that map code functions to corresponding design snippets. (See reference) Figure 3F This provides examples of the identified entries in the mapping dictionary 353. For example, one or more such mappings can be stored via data storage 212. Storing such mappings facilitates their subsequent use (e.g., via code analyzer 240).

[0145] For code-to-design matching (e.g., as indicated in a mapping table), the code-to-design analyzer 230 can facilitate the analysis of such identified design data associated with the code snippet. In this regard, the code-to-design analyzer 230 can identify any inconsistencies between the matched code data and the design data. Inconsistencies indicate that the code and design are not aligned, thus indicating that the code implementation is not based on the design. Any such inconsistencies can be identified and provided as output data.

[0146] In some cases, LLM or other AI techniques can be used to perform this type of analysis. In this regard, where matching design data fragments associated with code snippets (e.g., code functions) are identified, the data can be analyzed to assess any inconsistencies between the data. In this way, the design-to-code analyzer 240 can generate design-to-code analysis hints that are used to analyze the similarities and / or differences between the design data and the matching code portions. In some cases, design-to-code analysis hints can be generated for a specific design-to-code match. In other cases, design-to-code analysis hints can be generated for a set of design-to-code matches. Design-to-code analysis hints can include design data and / or code data and / or indications of such data as context. For example, design-to-code analysis hints can include identified design-to-code matches, or include references or indications of design-to-code matches for performing analysis. Thus, the analysis of design and / or code data is minimized to previously identified design-to-code matches, resulting in more efficient analysis.

[0147] It should be understood that, in other cases, the design involves code analyzer 240 additionally or alternatively identifying code snippets (e.g., functions) mapped to or matching design features. For example, for a particular design feature, the source code represented in the data store can be searched to identify code snippets that implement the functionality of that particular design feature.

[0148] Figure 3G An example hint is provided. Figure 3G In this context, input variables include one or more design-to-implementation mappings (354). For example... Figure 3G As shown, the prompts include instructions 355 related to iterating through each mapping, instructions 356 related to analyzing design fragments, instructions 357 related to analyzing one or more corresponding implementations, instructions 358 related to comparing designs with one or more implementations, instructions 359 related to inconsistencies in labeling, target output 360, and example output 361.

[0149] Design-code analysis hints can be provided as input to an LLM to perform analysis of design data associated with code data. Specifically, design data and corresponding or matching code data can be analyzed to identify any design-related inconsistencies or code gaps between the matching data (indicating design-code gaps or code that does not function correctly relative to the design). Safety results or outputs can include various types of information. As an example, an indication of the existence of inconsistencies can be provided. As another example, an indication of the design data and / or code data being analyzed can be provided. As yet another example, the degree of likelihood of inconsistencies can be provided. As yet another example, the degree of risk of not identifying inconsistencies (e.g., if not properly addressed) can be provided. Safety results can be provided at any level of granularity. For example, a general indication of design-to-code inconsistencies can be provided. As another example, an explanation of why design-to-code inconsistencies exist can be provided. Such safety results can be stored and / or provided for later use.

[0150] Although described as a two-step prompt generation and LLM analysis process (e.g., identifying relevant data and then using it), the implementation can include any number of steps (e.g., a single prompt generation and analysis step). Furthermore, while the examples perform this two-step process for each design-code pair (e.g., design-code pairs in the resulting set of design-code pairs), it is also possible to perform this process for a set of design-code matches or a batch of design-code matches.

[0151] Code analyzer 250 is typically configured to analyze code associated with a product or parts thereof to identify associated security risks. In this way, security risk analysis can be performed on the code even in instances where the design is suitable for the product and implemented according to such a suitable design. In various embodiments, code analyzer 250 can analyze code to identify potential security risks based on the programming language or implementation used to create the software product. For example, dangerous modes (e.g., direct memory access) may arise from code generation based on encoding or programming language. As an example only, suppose a particular programming language includes direct memory access. In this case, the developer may inadvertently introduce security problems associated with the developed code. Thus, code analyzer 250 can analyze code and programming language-specific aspects to identify any security risks.

[0152] In various embodiments, code analyzer 250 may perform data flow analysis, code analysis using PS data and / or OS data, and / or test coverage analysis to facilitate code analysis to identify security risk data. Data flow analysis generally refers to analyzing data flows associated with code to identify one or more security risks. Specifically, for data flow analysis, code analyzer 250 analyzes the inputs of source code functions and how these inputs are subsequently used in the function logic. In this way, code analyzer 250 can identify data flows and interfaces exposed to users to identify potential security issues. As an example only, assume the intention is to read 10 bytes of memory. However, it is also assumed that the user can input to read 100 bytes of memory. In this case, the code might read more data than expected, which could lead to potential security risks.

[0153] First, to perform data flow analysis, code can be analyzed to identify specific code snippets, thereby facilitating further security issue analysis. In this way, code corresponding to potential user access or interfaces can also be analyzed. In some cases, to identify code snippets, a set of code functions can be analyzed. As an example, a design-code map set can be analyzed to identify code functions that include interfaces exposed to users. In this regard, code analyzer 250 can access (e.g., generated via design-to-code analyzer 240) the design-code map to identify functions that correspond to data inputs and / or outputs that expose data to users, potentially introducing security risks that should be further evaluated. In some embodiments, design data and / or threat data can be used to facilitate the identification of data flows and interfaces exposed to users. For example, design data and / or threat data can indicate or include user interface exposures. Identifying functions that expose interfaces or data to users is valuable for performing efficient data flow analysis. For example, various code functions may not include any interface or user exposure, thus eliminating the need for further evaluation of potential security risks.

[0154] In some cases, code analyzer 250 uses AI techniques to facilitate the identification of code functions for further evaluation. For example, code analyzer 250 can facilitate the generation of code interface hints to identify code snippets, such as functions, that may expose security risks (e.g., interfaces). As an example only, code analyzer 250 can (e.g., via design-code mapping) access code functions and generate hints to analyze them to identify the data input to the function, how the function uses it, and / or the data output by the function. Figure 3H An example hint is provided. Figure 3H In this context, input variables can include design documents 362, threat models 363, and project data flows 364. For example... Figure 3HAs shown, the prompts include instructions 365 related to the analysis design document, instructions 366 related to the analysis threat model, instructions 367 related to the analysis project data flow, instructions 368 related to the identification interface, instructions 369 related to the marking of potential risks, target output 370, and example output 371.

[0155] Code interface hints can be input into an LLM to identify the interface or data exposed to the user. In response, the LLM can provide output indicating such requested information. For example, an LLM response may include security risk data indicating whether the function input, the type of one or more operations associated with the input, and / or whether the data is trusted or untrusted. In some cases, such hints may be specific to a particular code function. In other cases, hints may include various code functions.

[0156] Based on the identified functions requiring further analysis (e.g., based on potential interfaces or data exposed to users, or other security issues), code analyzer 250 can analyze functions with potential security problems. As an example only, suppose the function's implementation uses untrusted data to perform specific operations, such as memory access. In this case, the data flow associated with that function could be identified as having a dangerous pattern and therefore potentially a security issue.

[0157] In various embodiments, the code analyzer 250 may use AI techniques such as LLM to facilitate the identification of security risks. For example, LLM may be used to analyze whether the data flow associated with a function corresponds to or matches a security risk pattern identified as a potential security problem. In this regard, one or more code analysis hints may be generated to identify whether the data flow associated with a function matches a security risk pattern. A security risk pattern may include any known pattern that indicates a security vulnerability. In various embodiments, such a security risk pattern may correspond to specific programming code used to create or develop a product. An example of a security risk pattern may reflect one or more memory corruptions. Memory corruption typically refers to instances where a program unintentionally modifies memory (e.g., buffer overflow, use after free, heap corruption, etc.). Another example of a security risk pattern is an integer overflow. An integer overflow occurs when an arithmetic operation results in a value exceeding the maximum (or minimum) value that the data type can hold, which may produce unexpected values. Other examples of security vulnerabilities that may arise due to the characteristics and / or features of a programming language may include SQL injection, cross-site scripting, cross-site request forgery, command injection, path traversal, buffer overflow, use after free, null pointer dereference, deserialization vulnerabilities, etc. This security risk pattern can be analyzed individually or as a set. For example, as an example, analysis of memory operations (e.g., to identify the presence of potential memory corruption) can be performed to determine whether the data stream generated by the first hint can be used for any memory-related operations, and then integer overflow analysis can be performed.

[0158] Figure 3I An example hint is provided. Figure 3I In this context, input variables can include function source code 372 and security checklist 373. For example... Figure 3I As shown, the prompts include instructions 374 related to analyzing the source code of the function, instructions 375 related to reviewing the security checklist, instructions 376 related to performing security analysis, instructions 377 related to identifying security issues, target output 378, and example output 379. Figure 3I The document also includes an example safety checklist 380.

[0159] In response to code analysis hints input into an LLM, the LLM can provide security risk data as output, representing one or more potential security risks associated with the product code. This security risk data can include any type of information related to the security risk and can be provided in any number of formats. In one example, the output security risk data could indicate whether a function exposing an interface to the user is related to a security risk pattern. In another example, the security risk data could indicate a code snippet in which data from an incoming function is used in a memory copy operation. Figure 3JExamples of security analysis report 381 and tag status 382 are provided.

[0160] As described above, the code analyzer 250 can also perform code analysis using PS data and / or OS data. In this case, code can be analyzed to identify or determine whether the code contains one or more specific security risk patterns identified via PS data and / or OS data. In this case, code functions can be compared with security risk patterns generated via PS data and OS data. Security risk patterns can be general patterns generated or extracted based on PS data or OS data. In other cases, security risk patterns can be examples of data (e.g., code) containing security risks. In various embodiments, AI techniques such as LLM can be used to determine whether one or more code functions exhibit or match one or more security risk patterns. In this way, similarity between code functions and security risk patterns can be identified via prompts input to the LLM. If similarity or degree of similarity is identified therein, the security risk pattern can be identified as a match. Specific security risk patterns generated from the analyzed PS data and / or OS data can be predetermined, dynamically determined (e.g., based on associated code or product, etc.), etc.

[0161] Figure 3K An example hint is provided. Figure 3K In this context, input variables can include function source code (383) and vulnerability mode (384). For example... Figure 3I As shown, the prompt includes instructions 385 related to analyzing the source code of the function, instructions 386 related to reviewing the vulnerability pattern, instructions 387 related to performing vulnerability analysis, instructions 388 related to identifying the vulnerability pattern, target output 389, and example output 390. Figure 3L Example vulnerability pattern 391 is provided in the document.

[0162] In response to code analysis hints input into the LLM, the LLM can provide security risk data as output, representing one or more potential security risks associated with the product code, based on security risk patterns generated via PS data and / or OS data. This security risk data can include any type of information related to the security risk and can be provided in any number of formats. In one example, the output security risk data may indicate whether a security risk pattern matches or is similar to code or a code function. In another example, the security risk data may indicate the similarity between code or portions thereof and a corresponding security risk pattern generated via PS data and / or OS data. Figure 3M The document provides an example of vulnerability analysis report 392 and tag status 393.

[0163] Code analyzer 250 may alternatively or additionally perform test coverage analysis to facilitate code analysis. In this regard, code analyzer 250 may identify whether one or more security tests cover or correspond to code segments (e.g., functions). Security testing generally refers to any test used to identify or verify security risks. Examples of security tests include unit tests, integration tests, fuzzing, etc. Unit testing is typically a more granular level of testing that focuses on individual units of code (e.g., functions or methods) to ensure that these units perform as expected in isolation. Unit tests can be used to verify whether a unit of code (e.g., a function) can operate correctly on its own. Integration tests evaluate the interactions between multiple units of code to ensure that they work together correctly. Integration tests can verify whether different modules or services are correctly integrated and ensure that components produce the expected results when combined. Fuzzing typically provides software with random, unexpected, or invalid input to discover security vulnerabilities, crashes, and / or unexpected behavior. Fuzzing can identify edge cases and potential security vulnerabilities that standard tests might miss and ensure that the software can properly handle unexpected or malformed input. By using these different types of test coverage functions, developers can ensure that software is robust, reliable, and secure. In this way, performing test coverage analysis provides valuable insights into potential security risks associated with the code or parts thereof.

[0164] In some cases, the code analyzer 250 can identify whether any security tests cover a code segment based on the identification of security risks associated with that segment. For example, suppose a security risk is identified associated with data flow analysis of a specific code function or analysis of PS data and / or OS data. In this case, the code analyzer 250 can identify whether security tests are being performed on the code function. For example, this information can be indicated to the user, which may be valuable information related to the security of the code. For example, if no security tests associated with the code function are identified, the user can understand why the security issue exists. On the other hand, if security tests associated with the code function are identified, the user can identify or evaluate why the security tests failed to identify the security issue.

[0165] Code analyzer 250 can identify and / or capture data or statistics associated with various code-related analyses. For example, code analyzer 250 can detect or identify several security risks associated with data flow analysis and / or analysis of PS data and / or OS data, several functions associated with security risks, several functions covered by unit tests, several functions covered by integration tests, several functions covered by fuzzing tests, etc.

[0166] The various component security analysis managers 212 can operate in various ways. In some cases, the security analysis manager 212 can perform various analyses sequentially or concurrently. For example, in some cases, design analysis, design-to-code analysis, and code analysis can be performed sequentially (e.g., in a predetermined order). In other cases, product-related design analysis, design-to-code analysis, and / or code analysis can be performed concurrently. Furthermore, depending on the implementation, it may not be necessary to perform every type of analysis. For example, one or more specific types of product-related analyses performed may depend on one or more types of data obtained or available to the security analysis manager 212.

[0167] Results provider 260 is typically configured to provide analysis results, such as security risk data 292, identified through output data 290 of design analysis, design-to-code analysis, and / or code analysis. In this way, any indication of a security risk or associated data can be provided. In some cases, security risk data is provided to users, such as users who request to perform security analysis and / or view such data. In this way, security risk data can be provided to user devices via a network for display to users. Alternatively or additionally, security risk data can be provided to another component or resource for storage and / or further analysis or utilization. As an example, after a security risk is identified, the associated data can be provided to another component that facilitates code modifications to overcome or patch the security risk.

[0168] As described, security risk data may include any representation or indication of a security risk. Examples of security risk data include indications of code locations (e.g., lines) corresponding to potential security risks, code snippets associated with security risks, risk scores indicating the likelihood and / or severity of a security risk, explanations or causes related to the severity or likelihood of a security risk, analyses performed to identify potential security risks, and sources used to identify potential security risks (e.g., threat data, security requirements, etc.).

[0169] Solution Manager 270 is typically configured to manage solutions associated with security risks. In this regard, Solution Manager 270 can manage solutions to security risks identified through design analysis, design-to-code analysis, and / or code analysis. In various embodiments, Solution Manager 270 can facilitate the generation of solutions for one or more security risks. Therefore, for security risks associated with code, Solution Manager 270 can facilitate the generation of code solutions; while for security risks associated with design, Solution Manager 270 can facilitate the generation of design solutions. In this way, security risk solutions can be automatically generated to reduce or remediate potential security risks.

[0170] In some embodiments, AI can be used to facilitate the automatic generation of security risk solutions. As an example, an LLM (Local Management System) can be used to obtain hints and, in response, provide security risk solutions that address or mitigate security risks. For instance, based on security risks detected in code and their associated context (e.g., identification of the security risk, location of the risk, function relationships, etc.), an LLM can be used to generate fixes or patches for portions of code containing the security risk. In this way, solution identification hints can be generated to be input into the LLM. Such solution identification hints may include (e.g., security risks detected or identified by the security analysis manager 212), code or design data, or related portions associated with the security risk (e.g., code containing the security risk and other functions related to such code), code relationships (e.g., dependencies and relationships between functions), etc. Code relationships can assist the LLM in understanding the order in which functions are called and in better understanding the code, thereby facilitating the identification of more appropriate solutions. An example of providing code relationships may include providing a call graph indicating a sequence of functions (e.g., function A calls function B, etc.). The call graph can model the inter-process control flow in a way that visually represents the flow between methods. In some cases, previously generated call graphs can be obtained from a data store. In other cases, a call graph associated with the code can be generated.

[0171] Figure 3N An example hint is provided. Figure 3N In this context, input variables may include function source code 394, vulnerability details (e.g., security risks) set 395, function context 396 (e.g., call graph, dependencies, and / or data types used in the function), and source code repository 397. For example... Figure 3N As shown, the prompt includes various instructions (398) related to analyzing function source code, understanding vulnerabilities, analyzing function context, searching for other instances, and generating patches. Figure 3N As shown, an example output of 399 is also provided.

[0172] In some cases, solutions to one or more identified security risks can be automatically used to test code and / or design. For example, suppose an LLM generates or produces a code solution. In this case, the code solution can automatically replace one or more parts of code with security risks. Thus, Solution Manager 270 can replace the parts associated with the security risks with the generated solution and initiate testing. For example, suppose a code patch is generated for security risks identified in the code. The affected code with the identified security risks can be replaced with the code patch, after which the code with the patch can be executed and / or analyzed (e.g., via Security Analysis Manager 212) to identify whether the security risks have been mitigated.

[0173] In various embodiments, the identified security risk solution 294 may be provided as output data 290. In some cases, the identified solution may be implemented automatically. For example, based on the identification and testing of the security risk solution, the solution may be automatically integrated with the code and / or design. In other cases, the identified solution may be recommended or provided to the user for review. For example, engineers or security analysts may review the security risk solution and choose whether to integrate it (e.g., by accepting or rejecting the implementation via a user interface).

[0174] Turn Figure 4 , Figure 4 Example flows are provided to facilitate the management of automated security analysis, according to embodiments described herein. Figure 4 As shown, various types of data can be provided as input 402 to data storage 404 (e.g., vector data storage). Examples of such data include design documents, code repositories, threat models, one or more security requirements, PS datasets, OS datasets, etc. Data stored in data storage 404 can be accessed and used to perform various analysis phases. For example, various data can be used to perform design analysis 406. As described, design analysis may include analyzing designs related to threat data, security requirement data, and / or PS / OS data. Various data may be additionally or alternatively used to perform design-to-code analysis 408. Design-to-code analysis may include performing design-to-code analysis, generating design-to-code mappings 412, and design-to-implementation analysis. Further, various data may be additionally or alternatively used to perform code analysis 410, which may include data flow analysis, code analysis using PS data and / or OS data, and / or test coverage analysis. Any of these analyses (e.g., design analysis, design-to-code analysis, and / or code analysis) can be used to generate output 414, which includes various security risks identified through different analyses.

[0175] Now for reference Figures 5-7 Each box of methods 500, 600, and 700 described herein includes a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be implemented using one or more processors that execute instructions stored in one or more memories. The method can also be embodied as computer-usable instructions stored on a computer storage medium. These methods can be provided by a standalone application, by a service or managed service (standalone or in combination with other managed services), as a microservice via an application programming interface (API), or as a plug-in to another product, to name a few. Additionally, as an example, regarding... Figure 1 and Figure 2 The systems described herein are for methods 500, 600, and 700. However, additionally or alternatively, the method may be performed by any one of the systems or any combination of systems, including but not limited to the systems described herein.

[0176] Figure 5 This is a flowchart of a method 500 for performing automated security analysis according to some embodiments of the present disclosure. At block B502, the method 500 includes: identifying design data representing the design of a software product via one or more machine learning models to analyze potential security risks associated with the software product. In some cases, the design data to be analyzed is identified by: inputting a prompt into a large language model in one or more machine learning models to identify design data related to security threats; and obtaining a representation of the design data related to security threats in response to the prompt. Additionally or alternatively, the design data to be analyzed is identified by: inputting a prompt into a large language model in one or more machine learning models to identify design data related to security requirements; and obtaining a representation of the design data related to security requirements in response to the prompt. Further, in some cases, the design data to be analyzed can be identified by: inputting a prompt into a large language model in one or more machine learning models to identify design data related to proprietary security patterns or open-source patterns; and obtaining a representation of the design data related to proprietary security patterns or open-source patterns in response to the prompt. In yet another instance, the design data to be analyzed can be identified by: inputting a prompt into a large language model within one or more machine learning models to identify design data associated with a code snippet; and obtaining a representation of the design data associated with the code snippet in response to the prompt.

[0177] At box B504, method 500 includes: (e.g., using one or more machine learning models) analyzing design data to identify potential security risks associated with the software product. In some cases, analyzing design data to identify potential security risks associated with the software product includes: using a large language model from one or more machine learning models to determine that the design data fails to mitigate security threats, fails to meet security requirements, and / or fails to resolve PS or OS patterns. In other cases, analyzing design data to identify potential security risks associated with the software product includes: using a large language model from one or more machine learning models to determine inconsistencies between the design data and code snippets identified as related to the design data.

[0178] At box B506, method 500 includes: presenting a representation of potential security risks associated with the software product. The representation of potential security risks can be presented in any number of ways via a user interface. For example, the representation of potential security risks can be presented in a report format or an alert format.

[0179] Figure 6 A flowchart of a method 600 for performing automated security analysis according to some embodiments of the present disclosure is provided. At block B602, the method 600 includes: using one or more machine learning models to identify code corresponding to a software product to analyze potential security risks associated with the software product. In various embodiments, the code includes code functions corresponding to potentially exposed interfaces. In some cases, the code to be analyzed is identified by: inputting a prompt into a large language model in one or more machine learning models to identify code corresponding to a potentially exposed interface; and obtaining a representation of the code corresponding to the potentially exposed interface in response to the prompt.

[0180] At box B604, method 600 includes: analyzing code using one or more machine learning models to identify potential security risks associated with a software product. In some cases, analyzing code to identify potential security risks associated with a software product includes: using a large language model from one or more machine learning models to determine that the data flow associated with the code corresponds to a representation of a security risk pattern, or that the code includes a security risk pattern generated based on proprietary or open-source security data.

[0181] At box B606, method 600 includes: presenting a representation of potential security risks associated with the software product. This presentation can be in any number of formats and is not intended to be limited to those described herein. In some cases, security risk solutions associated with identifying potential security risks can be identified or generated (e.g., via one or more ML models).

[0182] Turn Figure 7 , Figure 7 A flowchart illustrating a method 700 for performing automated security analysis according to some embodiments of the present disclosure is provided. At block B702, the method 700 includes: identifying design data representing the design of a software product to analyze one or more potential security risks associated with the software product.

[0183] At box B704, method 700 includes: providing a representation of design data as at least a portion of the input to one or more machine learning models to identify potential security risks associated with a software product. In some cases, the input includes a hint, and one or more machine learning models include a large language model. The hint may include representations of various other data, such as threat data, security requirement data, proprietary security data, or open-source data, to identify potential security risks associated with the software product. As another example, the hint may include a code representation corresponding to the design data to identify potential security risks associated with the software product.

[0184] The systems and methods described herein can be used for a variety of purposes, as examples but not limited to machine (e.g., robots, vehicles, construction machinery, warehouse vehicles / machines, autonomous, semi-autonomous and / or other machine types) control, machine motion, machine driving, synthetic data generation, model training (e.g., using real, augmented and / or synthetic data, such as synthetic data generated using simulation platforms or systems, synthetic data generation techniques (such as, but not limited to, those described herein), perception, augmented reality (AR), virtual reality (VR), hybrid implementation (MR), robotics, security and supervision (e.g., in smart city implementations), autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twins, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), distributed or collaborative content creation for 3D assets (e.g., using generic scene descriptor (USD) data, such as OpenUSD and / or other data types), cloud computing, generative artificial intelligence (e.g., using one or more diffusion models, converter models, etc.) and / or any other suitable application.

[0185] The disclosed embodiments can comprise a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots or robotic platforms, aviation systems, medical systems, rowing systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations (e.g., in driving or vehicle simulations, in robot simulations, in smart city or supervised simulations, etc.), systems for performing digital twin operations (e.g., in conjunction with collaborative content creation platforms or systems, such as, but not limited to, NVIDIA's OMNIVERSE and / or another platform, system, or service using USD or OpenUSD data types), systems implemented using edge devices, systems containing one or more virtual machines (VMs), and systems for (e.g.) For example, systems that perform synthetic data generation operations using one or more neural rendering fields (NERF), Gaussian sputtering techniques, diffusion models, converter models, etc.; systems implemented at least partially in data centers; systems for performing conversational AI operations; systems that implement one or more language models (such as one or more large language models (LLM), one or more visual language models (VLM), one or more multimodal language models, etc.); systems for performing optical transport simulations; systems for performing collaborative content creation for 3D assets (e.g., using generic scene descriptor (USD) data, such as OpenUSD, computer-aided design (CAD) data, 2D and / or 3D graphics or design data and / or other data types); systems implemented at least partially using cloud computing resources; and / or other types of systems.

[0186] In some examples, one or more machine learning models described herein (e.g., deep neural networks, language models, LLM, VLM, multimodal language models, perceptual models, tracking models, fusion models, transformer models, diffusion models, encoder-only models, decoder-only models, encoder-decoder models, neural rendering field (NERF) models, etc.) can be packaged as microservices, such as inference microservices (e.g., NVIDIA NIM), which may include containers (e.g., operating system (OS) level virtualization packages) that may include an application programming interface (API) layer, a server layer, a runtime layer, and / or a model "engine". For example, an inference microservice may include the container itself and one or more models (e.g., weights and biases). In some instances (e.g., where one or more machine learning models are small enough (e.g., have a sufficiently few parameters)), one or more models may be included within the container itself. In other instances (e.g., where one or more models are large), one or more models may be hosted / stored in the cloud (e.g., in a data center) and / or hosted on-premises and / or at the edge (e.g., on a local server or compute device, but outside the container). In these embodiments, one or more models can be accessed via one or more APIs, such as REST APIs. Thus, and in some embodiments, one or more machine learning models described herein can be deployed as inference microservices to accelerate the deployment of one or more models on any cloud, data center, or edge computing system while ensuring data security. For example, an inference microservice may include one or more APIs, pre-configured containers for simplified deployment, an optimized inference engine (e.g., execution software built using standardized AI model deployments, such as NVIDIA's Triton Inference Server), and / or one or more APIs for high-performance deep learning inference, which may include inference runtime and model optimizations for delivering low latency and high throughput for production applications (such as NVIDIA's TensorRT), and / or enterprise management data for telemetry (e.g., including identity, metrics, health checks, and / or monitoring). One or more machine learning models described herein may be included as part of a microservice along with an acceleration infrastructure capable of deployment using a single command and / or orchestration and autoscaling using a container orchestration system on the acceleration infrastructure (e.g., reaching data center scale on a single device). Thus, inference microservices may include machine learning models (e.g., those optimized for high-performance inference), inference runtime software that executes one or more machine learning models and provides responses / outputs to inputs (e.g., user queries, prompts, etc.), and enterprise management software that provides health checks, identity and / or other monitoring.In some embodiments, the inference microservice may include software that performs in-situ replacement and / or updates to one or more machine learning models. When a replacement or update is performed, the software performing the replacement / update may maintain user configurations for the inference runtime software and the enterprise management software.

[0187] Example language model

[0188] In at least some embodiments, language models such as Large Language Models (LLM), Visual Language Models (VLM), Multimodal Language Models (MMLM), and / or other types of generative artificial intelligence (AI) can be implemented. These models may be able to understand, summarize, translate, and / or otherwise generate text (e.g., natural language text, code, etc.), images, videos, computer-aided design (CAD) assets, OMNIVERSE and / or METAVERSE file information (e.g., USD formats such as OpenUSD), and / or the like based on context provided in input prompts or queries. In embodiments, these language models may be considered “large” because they are trained on massive datasets and have architectures with a large number of learnable network parameters (weights and biases)—e.g., millions or billions of parameters. LLM / VLM / MMLM / etc. can be implemented for summarizing textual data, analyzing data (e.g., text, images, videos, etc.), extracting insights from data (e.g., text, images, videos, etc.), and generating new text / images / videos / etc. in a user-specified style, tone, and / or format. In some embodiments, the LLM / VLM / MMLM / etc. disclosed herein may be specifically designed for text processing, while in others, a multimodal LLM may be implemented to accept, understand, and / or generate text and / or other types of content, such as images, audio, 2D and / or 3D data (e.g., USD format) and / or video. For example, a Visual Language Model (VLM) or more specifically a Multimodal Language Model (MMLM) may be implemented to accept images, video, audio, text, 3D designs (e.g., CAD) and / or other input data types and / or generate or output images, video, audio, text, 3D designs and / or other output data types.

[0189] Various types of LLM / VLM / MMLM / etc. architectures can be implemented in various embodiments. For example, different architectures can be implemented using different techniques to understand and generate outputs (e.g., text, audio, video, images, 2D and / or 3D design or asset data, etc.). In some embodiments, LLM / VLM / MMLM / etc. architectures (e.g., recurrent neural networks (RNNs) or long short-term memory networks (LSTMs)) can be used, while in other embodiments, converter architectures (e.g., architectures relying on self-attention and / or cross-attention (e.g., between contextual data and textual data) mechanisms) can be used to understand and recognize relationships between words or tokens and / or contextual data (e.g., other text, video, images, design data, USD, etc.). One or more generative processing pipelines including LLM / VLM / MMLM / etc. may also include one or more diffusion blocks (e.g., noise reduction blocks). The LLM / VLM / MMLM / etc. of this disclosure may include encoder and / or decoder blocks. For example, discriminative or encoder-only models (e.g., BERT (Bidirectional Encoder Representations from Transformers)) can be implemented for tasks involving language understanding (e.g., classification, sentiment analysis, question answering, and named entity recognition). As another example, generative or decoder-only models (e.g., GPT (Generative Pretrained Transformer)) can be implemented for tasks involving language and content generation (e.g., text completion, story generation, and dialogue generation). LLM / VLM / MMLM / etc., including encoder and decoder components (e.g., T5 (Text-to-Text Transformer)), can be implemented to understand and generate content, such as for translation and summarization. These examples are not intended to be limiting and any architecture type (including, but not limited to, those described herein) can be implemented depending on the specific implementation and the task performed using LLM / VLM / MMLM / etc.

[0190] In various embodiments, LLM / VLM / MMLM / etc. can be trained using unsupervised learning, whereby LLM / VLM / MMLM / etc. learns patterns from a large amount of unlabeled text / audio / video / image / design / USD / etc. data. Due to the extensive training, in these embodiments, the model may not require task-specific or domain-specific training. An LLM / VLM / MMLM / etc. extensively pre-trained on a large amount of unlabeled data can be referred to as a base model and can excel at various tasks, such as question answering, summarizing, filling in missing information, translation, and image / video / design / USD / data generation. Some LLM / VLM / MMLM / etc. can be customized for specific use cases using techniques such as cue tuning, fine-tuning, retrieval augmentation generation (RAG), adding adapters (e.g., custom neural networks and / or neural network layers to tune or adjust cues or labels to bias the language model towards a specific task or domain), and / or using optimization models for specific tasks and / or other fine-tuning or customization techniques within a specific domain.

[0191] In some embodiments, the LLM / VLM / MMLM / etc. disclosed herein can be implemented using various model alignment techniques. For example, in some embodiments, guardrails can be implemented to identify incorrect or unwanted inputs (e.g., prompts) and / or outputs of the model. In this process, the system can use guardrails and / or other model alignment techniques to prevent the processing of specific unwanted inputs using LLM / VLM / MMLM / etc., and / or to prevent the output or presentation of information generated by LLM / VLM / MMLM / etc. (e.g., displays, audio outputs, etc.). In some embodiments, one or more additional models (or layers thereof) can be implemented to identify problems with the model's inputs and / or outputs. For example, these "protective" models can be trained to identify "safe" or otherwise okay or desired inputs and / or outputs and / or "unsafe" or otherwise unwanted inputs and / or outputs for a particular application / implementation. Therefore, the LLM / VLM / MMLM / etc. disclosed herein are unlikely to output language / text / audio / video / design data / USD data / etc. that may be offensive, vulgar, inappropriate, insecure, out of scope, and / or unwanted for a particular application / implementation.

[0192] In some embodiments, an LLM / VLM / etc. can be configured or able to access or use one or more plugins, application programming interfaces (APIs), databases, data stores, repositories, etc. For example, for certain tasks or operations where the model is not ideally suited, the model may have instructions for accessing one or more plugins (e.g., third-party plugins) to help process the current input (e.g., as a result of training, and / or based on instructions in a given prompt). In such an example, when at least part of the prompt relates to restaurants or weather, the model can access one or more restaurant or weather plugins (e.g., via one or more APIs) to retrieve relevant information. Another example is that if at least part of the response requires mathematical computation, the model can access one or more mathematical plugins or APIs to help solve the problem, and then the response from the plugins and / or APIs can be used in the model's output. This process can be repeated (e.g., recursively) an arbitrary number of iterations, using any number of plugins and / or APIs, until a response to each query / question / request / process / action / etc. can be generated in response to the input prompt. Therefore, models can rely not only on their own knowledge gained from training on large datasets, but also on the expertise or optimized properties of one or more external resources (such as APIs, plugins, etc.).

[0193] In some embodiments, multiple language models (e.g., LLM / VLM / MMLM / etc., multiple instances of the same language model, and / or multiple hints provided to the same language model or instances of the same language model) can be implemented, executed, or accessed (e.g., using one or more plugins, user interfaces, APIs, databases, data stores, repositories, etc.) to provide output in response to the same query or in response to separate parts of a query. In at least one embodiment, the same input query and hints (e.g., a set of constraints, condition generators, etc.) can be provided to multiple language models (e.g., language models with different architectures, language models trained on different (e.g., updated) data corpora). In one or more embodiments, the language models can be different versions of the same base model. In one or more embodiments, at least one language model can be instantiated as multiple agents, for example, providing more than one hint to constrain, guide, or otherwise influence the style, content, or character of the provided output. In one or more exemplary non-limiting embodiments, the same language model can be required to provide output corresponding to different roles, perspectives, characters, or different knowledge bases, as defined by the provided hints.

[0194] In any such embodiment, the outputs of two or more (e.g., each) language models, two or more versions of at least one language model, two or more instantiated proxies of at least one language model, and / or provided to two or more prompts for at least one language model can be further processed, such as aggregated, compared, or filtered, or used to determine (and provide) a consensus response. In one or more embodiments, the output from one language model (or version, instance, or proxy) can be provided as input to another language model for further processing and / or validation. In one or more embodiments, the language model can be required to generate or otherwise obtain output about the input source material, wherein the output is associated with the input source material. This association may include, for example, generating captions or text portions embedded (e.g., as metadata) within the input source text or image. In one or more embodiments, the output of the language model can be used to determine the validity of the input source material for further processing or inclusion in a dataset. For example, the language model can be used to evaluate the presence (or absence) of a target word in a text portion or the presence (or absence) of an object in an image, wherein the text or image is annotated to indicate such presence (or absence). Alternatively, the determination from the language model can be used to determine whether the source material should be included in the curatorial dataset, for example, but not limited to this.

[0195] Figure 8A This is a block diagram of an example generative language model system 800 suitable for implementing at least some embodiments of the present disclosure. Figure 8A In the example shown, the generative language model system 800 includes a retrieval augmentation generation (RAG) component 892, an input processor 805, a tokenizer 810, an embedding component 820, a plug-in / API 895, and a generative language model (LM) 830 (which may include LLM, VLM, multimodal LM, etc.).

[0196] At a high level, the input processor 805 can receive input 801, which includes text and / or other types of input data (e.g., audio data, video data, image data, sensor data (e.g., LiDAR, RADAR, ultrasound, etc.), 3D design data, CAD data, generic scene descriptor (USD) data (e.g., OpenUSD, etc.), depending on the architecture of the generative LM 830 (e.g., LLM / VLM / MMLM, etc.). In some embodiments, input 801 includes plain text in the form of one or more sentences, paragraphs, and / or documents. Additionally or alternatively, input 801 may include numerical sequences, pre-computed embeddings (e.g., word or sentence embeddings), and / or structured data (e.g., tabular format, JSON, or XML). In generative LM In some implementations of 830 capable of handling multimodal input, input 801 can combine text (or text that may be omitted) with image data, audio data, video data, design data, USD data, and / or other types of input data (e.g., but not limited to the data described herein). Taking raw input text as an example, input processor 805 can prepare the raw input text in various ways. For example, input processor 805 can perform various types of text filtering to remove noise from relevant text content (e.g., special characters, punctuation marks, HTML tags, stop words, portions of images, portions of audio, etc.). In examples involving stop words (common words that often have little semantic meaning), input processor 805 can remove stop words to reduce noise and allow the generative LM 830 to focus on more meaningful content. Input processor 805 can apply text normalization, for example, by converting all characters to lowercase, removing accent marks, and / or handling special cases (such as abbreviations or abbreviations) to ensure consistency. These are just a few examples; other types of input processing can be applied.

[0197] In some embodiments, RAG component 892 (which may include one or more RAG models, and / or may be performed using generative LM 830 itself) may be used to retrieve additional information to be used as part of input 801 or a prompt. RAGs can be used to enhance input to LLM / VLM / MMLM / etc. with external knowledge to make the answer to a specific question or query or request more relevant, for example, where specific knowledge is required. RAG component 892 may obtain this additional information from one or more external sources (e.g., basic information such as basic text / images / videos / audio / USD / CAD / etc.), which can then be fed along with the prompt to LLM / VLM / MMLM / etc. to improve the accuracy of the model's response or output.

[0198] For example, in some embodiments, in addition to the data retrieved using RAG component 892, input 801 may also be generated using query or model input (e.g., questions, requests, etc.). In some embodiments, input processor 805 may analyze input 801 and communicate with RAG component 892 (or in embodiments, RAG component 892 may be part of input processor 805) to identify relevant text and / or other data to provide to generative LM 830 as additional context or information source, typically from which to identify responses, answers, or outputs 890. For example, when the input indicates that a user is interested in the required tire pressure for a particular brand and model of vehicle, RAG component 892 may use a RAG model, for example, to perform a vector search in the embedding space to retrieve tire pressure information or its corresponding text from a digital (embedded) version of the owner's manual for that particular vehicle brand and model. Similarly, when a user revisits the chatbot related to a specific product sale or service, the RAG component 892 can retrieve previously stored conversation history (or at least its summary) and provide the previous conversation history, along with the current inquiry / request, as part of the generative LM 830 as input 801.

[0199] RAG component 892 can use various RAG techniques. For example, it can use naive RAG. The document is indexed, chunked, and applied to an embedding model to generate embeddings corresponding to chunks. User queries can also be applied to this embedding model and / or another embedding model of the RAG component 892, and the embeddings of the chunks can be compared with the embeddings of the query to identify the most similar / relevant embeddings to the query. These most similar / relevant embeddings can be provided to the generative LM 830 to generate output.

[0200] In some embodiments, more advanced RAG techniques can be used. For example, chunks can undergo pre-retrieval processes (e.g., routing, rewriting, metadata analysis, expansion, etc.) before being passed to the embedding model. Furthermore, post-retrieval processes (e.g., re-ranking, hint compression, etc.) can be performed on the output of the embedding model before generating the final embedding, which is then used for comparison with the input query.

[0201] As a further example, modular RAG techniques can be used, such as those similar to Naive RAG and / or Advanced RAG, but also including features such as hybrid search, recursive retrieval and query engines, StepBack methods, subqueries and hypothetical document embeddings.

[0202] As another example, Graph RAG can use a knowledge graph as a source of context or factual information. Graph RAG can be implemented using a graph database as a source of contextual information sent to LLM / VLM / MMLM / etc. Instead of providing the model with data chunks extracted from larger documents (which may result in a lack of context, factual accuracy, linguistic accuracy, etc.) (or anything other than providing the model with data chunks extracted from larger documents), Graph RAG can also provide structured entity information to LLM / VLM / MMLM / etc. by combining structured entity text descriptions with their many attributes and relationships, thus giving the model deeper insights. In implementing Graph RAG, the systems and methods described herein use graphs as content stores and extract relevant document chunks, requiring LLM / VLM / MMLM / etc. to use them to answer questions. In such embodiments, the knowledge graph may contain relevant textual content and metadata about the knowledge graph, or it may be integrated with a vector database. In some embodiments, Graph RAG can use the graph as a subject matter expert, where descriptions of concepts and entities relevant to the query / hint can be extracted and passed to the model as semantic context. These descriptions may include relationships between concepts. In other examples, the graph can be used as a database where a portion of a query / hint can be mapped to a graph query, the graph query can be executed, and LLM / VLM / MMLM / etc. can aggregate the results. In such examples, the graph can store relevant factual information and can be used for queries (natural language queries) and entity links to graph query tools (NL to graph query tools). In some embodiments, the graph RAG (e.g., using a graph database) can be combined with standard (e.g., vector database) RAGs and / or other RAG types to benefit from a variety of approaches.

[0203] In any embodiment, the RAG component 892 can implement plugins, APIs, user interfaces, and / or other functions to perform RAG. For example, LLM / VLM / MMLM / etc. can use graph RAG plugins to run queries on knowledge graphs to extract relevant information to feed into the model, and can use standard or vector RAG plugins to run queries on vector databases. For example, the graph database can interact with the plugin's REST interface, thus decoupling the graph database from the vector database and / or the embedded model.

[0204] The tokenizer 810 can segment (e.g., processed) text data into smaller units (tags) for subsequent analysis and processing. Depending on the implementation, the tags can represent individual words, sub-words, characters, audio / video / images, etc. Word-based tokenization divides the text into individual words, treating each word as a separate tag. Sub-word tokenization breaks words down into smaller meaningful units (e.g., prefixes, suffixes, stems), enabling the generative LM 830 to understand morphological changes and process words outside the vocabulary more effectively. Character-based tokenization represents each character as a separate tag, enabling the generative LM 830 to process text at a fine-grained level. The choice of tokenization strategy can depend on factors such as the language being processed, the task at hand, and / or the characteristics of the training dataset. Therefore, the tokenizer 810 can transform (e.g., processed) text into a structured format according to the tokenization scheme implemented in a particular embodiment.

[0205] Embedding component 820 can use any known embedding technique to transform discrete tokens into semantically meaningful (e.g., dense, continuous vector) representations. For example, embedding component 820 can use pre-trained word embeddings (e.g., Word2Vec, GloVe, or FastText), one-hot encoding, Term Frequency-Inverse Document Frequency (TF-IDF) encoding, one or more embedding layers of a neural network, and / or others.

[0206] In some implementations where input 801 includes image data / video data, etc., input processor 801 may resize the data to a standard size compatible with the format of the corresponding input channel and / or normalize pixel values ​​to a common range (e.g., 0 to 1) to ensure consistent representation, and embedding component 820 may encode the image data using any known technique (e.g., using one or more convolutional neural networks (CNNs) to extract visual features). In some implementations where input 801 includes audio data, input processor 801 may resample the audio file to a consistent sampling rate for uniform processing, and embedding component 820 may use any known technique to extract and encode audio features, such as in the form of a spectrogram (e.g., a Mel spectrogram). In some implementations where input 801 includes video data, input processor 801 may extract frames or apply resizing to extracted frames, and embedding component 820 may extract features such as optical flow embedding or video embedding and / or encode temporal information or frame sequences. In some implementations where input 801 includes multimodal data, the embedded component 820 can use techniques such as early fusion (concatenation), late fusion (sequential processing), and attention-based fusion (e.g., self-attention, cross-attention) to fuse representations of different types of data (e.g., text, images, audio, data, video, design, etc.).

[0207] Other components of the generative LM 830 and / or generative LM system 800 may use different types of neural network architectures depending on the implementation scheme. For example, a transducer-based architecture (such as the one used in models like GPT) may be implemented, and it may include a self-attention mechanism that weights the importance of different words or tokens in the input sequence and / or a feedforward network that processes the output of the self-attention layer, applying a nonlinear transformation to the input representation and extracting higher-level features. Some non-limiting example architectures include transducers (e.g., encoder-decoder, decoder-only, multimodal), RNNs, LSTMs, fusion models, diffusion models, cross-modal embedding models that learn a joint embedding space, graph neural networks (GNNs), hybrid architectures that combine different types of adversarial networks (such as generative adversarial networks or GANs or adversarial autoencoders (AAEs) for joint distribution learning), etc. Therefore, depending on the implementation scheme and architecture, the embedded component 820 can apply the encoded representation of the input 801 to the generative LM 830, and the generative LM 830 can process the encoded representation of the input 801 to generate an output 890, which may include response text and / or other types of data.

[0208] As described herein, in some embodiments, the generative LM 830 may be configured to access or use (or be able to access or use) plugins / APIs 895 (which may include one or more plugins, application programming interfaces (APIs), databases, data stores, repositories, etc.). For example, for certain tasks or operations where the generative LM 830 is not ideally suited, the model may have instructions (e.g., as a result of training, and / or based on instructions in a given prompt, such as instructions retrieved using RAG component 892) to access one or more plugins / APIs 895 (e.g., third-party plugins) to help process the current input. In such an example, when at least part of the prompt is related to a restaurant or weather, the model may access one or more restaurant or weather plugins (e.g., via one or more APIs), sending at least part of the prompt related to a particular plugin / API 895 to the plugin / API 895, which may process the information and return an answer to the generative LM 830, which may use the response to generate output 890. This process can be repeated (e.g., recursively) an arbitrary number of iterations and repeated with any number of plugins / APIs 895 until an output 890 that resolves each query / question / request / process / action / etc. from input 801 is generated. Therefore, the model can rely not only on its own knowledge gained from training on a large dataset and / or from data retrieved using the RAG component 892, but also on the expertise or optimized properties of one or more external resources (e.g., plugins / APIs 895).

[0209] Figure 8B This is a block diagram of an example implementation scheme, where the generative LM 830 includes a converter encoder-decoder. For example, suppose the input text (e.g., “Who discovered gravity”) is tokenized (e.g., by...) Figure 8A The tokenizer 810) is used for tokens such as words, and each token is encoded (e.g., by...). Figure 8A The embedding component 820 is a corresponding embedding (e.g., of size 512). Since these token embeddings do not typically represent the position of the tokens in the input sequence, positional encoding can be added to each token embedding using any known technique to encode the order relation and context of the tokens in the input sequence. Thus, (e.g., the resulting) embeddings can be applied to one or more encoders 835 of the generative LM 830.

[0210] In the example implementation, encoder 835 forms an encoder stack, where each encoder includes a self-attention layer and a feedforward network. In the example converter architecture, each token (e.g., a word) flows through a separate path. Therefore, each encoder can accept a sequence of vectors, pass each vector through the self-attention layer, then through the feedforward network, and then up to the next encoder in the stack. Any known self-attention technique can be used. For example, to compute a self-attention score for each token (word), a query vector, a key vector, and a value vector can be created for each token. The self-attention score for a token pair can be computed by taking the dot product of the query vector and the corresponding key vector, normalizing the resulting score, multiplying by the corresponding value vector, and summing the weighted value vectors. The encoder can apply multi-head attention, where the attention mechanism is applied multiple times in parallel with different learned weight matrices. Any number of encoders can be cascaded to generate a context vector encoding the input. The attention projection layer 840 can transform the context vector into an attention vector (key and value) for the decoder 845.

[0211] In the example implementation, decoder 845 forms a decoder stack, where each decoder includes a self-attention layer, an encoder-decoder self-attention layer that uses attention vectors (keys and values) from the encoder to focus on relevant parts of the input sequence, and a feedforward network. Similar to encoder 835, in the example converter architecture, each token (e.g., a word) flows through a separate path in decoder 845. During the first pass, decoder 845, classifier 850, and generation mechanism 855 can generate a first token, and generation mechanism 855 can apply the generated token as input during a second pass. This process can be repeated cyclically, generating tokens (e.g., words) and adding them to the output of the previous pass, and in subsequent passes applying token embeddings of positionally encoded composite sequences as input to decoder 845, generating one token at a time (called autoregression) until a symbol or token representing the end of the response is predicted. In each decoder, the self-attention layer is typically restricted to focusing only on preceding positions in the output sequence by applying a masking technique (e.g., setting future positions to negative infinity) before the softmax operation. In the example implementation, the encoder-decoder attention layer operates similarly to the (e.g., multi-head) self-attention operation in the encoder 835, except that it creates its queries from the layers below it and obtains keys and values ​​(e.g., matrices) from the output of the encoder 835.

[0212] Therefore, decoder 845 can output some decoded (e.g., vector) representation of the input applied during a particular pass. Classifier 850 can include a multi-class classifier comprising one or more neural network layers and a softmax operation that transforms logit probabilities into probabilities, the neural network layers projecting the decoded (e.g., vector) representation onto corresponding dimensions (e.g., one dimension for each supported word or token in the output vocabulary). Thus, generation mechanism 855 can select or sample words or tokens based on corresponding predicted probabilities (e.g., selecting the word with the highest predicted probability) and append it to the output of the previous pass, thereby generating each word or token sequentially. Generation mechanism 855 can repeat this process, triggering successive decoder inputs and corresponding predictions until a symbol or token representing the end of the response is selected or sampled, at which point generation mechanism 855 can output the generated response.

[0213] Figure 8C This is a block diagram of an example implementation where the generative LM 830 includes a decoder-only converter architecture. For example, Figure 8C The decoder 860 can be used with Figure 8B The decoder 845 operates similarly, except... Figure 8C Each decoder 860 omits the encoder-decoder self-attention layer (because there is no encoder in this implementation). Therefore, decoders 860 can form a decoder stack, where each decoder includes a self-attention layer and a feedforward network. Furthermore, instead of encoding the input sequence, a symbol or tag indicating the end of the input sequence (or the beginning of the output sequence) can be appended to the input sequence, and the resulting sequence (e.g., a corresponding embedding with positional encoding) can be applied to decoder 860. Figure 8B Similar to decoder 845, each tag (e.g., a word) can flow through a separate path in decoder 860, and decoder 860, classifier 865, and generation mechanism 870 can use autoregression to generate one tag at a time sequentially until a symbol or tag indicating the end of the response is predicted. Classifier 865 and generation mechanism 870 can be combined with... Figure 8B The classifier 850 and the generation mechanism 855 operate similarly, wherein the generation mechanism 870 selects or samples each consecutive output label based on the corresponding predicted probability and appends it to the output of the previous iteration, generating each label sequentially until a symbol or label representing the end of the response is selected or sampled. The architectures described herein, and others, are merely examples, and other suitable architectures may be implemented within the scope of this disclosure.

[0214] Example computing device

[0215] Figure 9This is a block diagram of an example computing device 900 suitable for implementing some embodiments of the present disclosure. The computing device 900 may include an interconnect system 902 directly or indirectly coupled to: a memory 904, one or more central processing units (CPUs) 906, one or more graphics processing units (GPUs) 908, a communication interface 910, input / output (I / O) ports 912, input / output components 914, a power supply 916, one or more presentation components 918 (e.g., one or more displays), and one or more logic units 920. In at least one embodiment, one or more computing devices 900 may include one or more virtual machines (VMs), and / or any of their components may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 908 may include one or more vGPUs, one or more CPUs 906 may include one or more vCPUs, and / or one or more logic units 920 may include one or more virtual logic units. Thus, one or more computing devices 900 may include discrete components (e.g., a full GPU dedicated to computing device 900), virtual components (e.g., a portion of the GPU dedicated to computing device 900), or a combination thereof.

[0216] although Figure 9 The various blocks are shown as connected via interconnect system 902 using lines, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, presentation component 918 (such as a display device) may be considered I / O component 914 (e.g., if the display is a touchscreen). As another example, CPU 906 and / or GPU 908 may include memory (e.g., memory 904 may represent a storage device other than the memory of GPU 908, CPU 906, and / or other components). Therefore, Figure 9 The computing devices described are for illustrative purposes only. No distinction is made between such categories as “workstation,” “server,” “laptop computer,” “desktop computer,” “tablet computer,” “client device,” “mobile device,” “handheld device,” “game console,” “electronic control unit (ECU),” “virtual reality system,” and / or other device or system types, as all are considered within the scope of… Figure 9 Within the scope of computing devices.

[0217] Interconnect system 902 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 902 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Fast Peripheral Component Interconnect (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 906 may be directly connected to memory 904. Further, CPU 906 may be directly connected to GPU 908. In cases where there is a direct or point-to-point connection between components, interconnect system 902 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required to be included in computing device 900.

[0218] The memory 904 may include any computer-readable medium from a variety of computer-readable media. A computer-readable medium may be any available medium accessible by the computing device 900. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0219] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 904 may store computer-readable instructions (e.g., representing one or more programs and / or one or more program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 900. As used herein, computer storage media does not include the signal itself.

[0220] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and includes any information transmission medium. The term "modulated data signal" can refer to a signal whose one or more characteristics are set or altered in a manner that encodes information in the signal. By way of example and not limitation, computer storage media can include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above should also be included within the scope of computer-readable media.

[0221] CPU 906 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 900 to perform one or more of the methods and / or processes described herein. Each CPU 906 may contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling numerous software threads simultaneously. CPU 906 may contain any type of processor and may contain different types of processors depending on the type of computing device 900 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 900, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors (such as math coprocessors), computing device 900 may also include one or more CPUs 906.

[0222] In addition to or in lieu of one or more CPUs 906, one or more GPUs 908 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 900 to perform one or more of the methods and / or processes described herein. One or more GPUs 908 may be integrated GPUs (e.g., having one or more CPUs 906) and / or one or more GPUs 908 may be discrete GPUs. In embodiments, one or more GPUs 908 may be coprocessors of one or more CPUs 906. GPUs 908 may be used by computing device 900 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, GPUs 908 may be used for general-purpose computing on a GPU (GPGPU). GPUs 908 may include hundreds or thousands of cores capable of handling hundreds or thousands of software threads simultaneously. GPUs 908 may generate pixel data for an output image in response to rendering commands (e.g., rendering commands received from CPUs 906 via a host interface). GPU 908 may include graphics memory (e.g., display memory) for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory may be included as part of memory 904. GPU 908 may include two or more GPUs operating in parallel (e.g., via links). The links may directly connect the GPUs (e.g., using NVLINK) or may connect the GPUs via a switch (e.g., using NVSwitch). When combined, each GPU 908 may produce pixel data or GPGPU data for different portions of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for an analog image). Each GPU may include its own memory or may share memory with other GPUs.

[0223] In addition to or in lieu of CPU 906 and / or GPU 908, logic unit 920 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 900 to perform one or more of the methods and / or processes described herein. In embodiments, one or more CPUs 906, one or more GPUs 908, and / or one or more logic units 920 may execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more logic units 920 may be a portion of one or more CPUs 906 and / or GPUs 908 and / or integrated into one or more CPUs 906 and / or GPUs 908, and / or one or more logic units 920 may be discrete components or otherwise external to CPUs 906 and / or GPUs 908. In embodiments, one or more logic units 920 may be coprocessors of one or more CPUs 906 and / or GPUs 908.

[0224] Examples of logic unit 920 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree lateral unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), programmable vision accelerator (PVA) (which may include one or more direct memory access (DMA) systems), one or more vision or vector processing units (VPU), and one or more pixel processing engines (PPE) (e.g.) Examples include 2D arrays of processing elements (each of which communicates north, south, east, and west with one or more other processing elements in the array), one or more decoupled accelerators or units (e.g., decoupled lookup table (DLUT) accelerators or units), vision processing units (VPUs), optical flow accelerators (OFAs), field-programmable gate arrays (FPGAs), neuromorphic chips, quantum processing units (QPUs), associative processing units (APUs), arithmetic logic units (ALUs), application-specific integrated circuits (ASICs), floating-point units (FPUs), input / output (I / O) elements, peripheral component interconnects (PCIs) or fast peripheral component interconnects (PCIe) elements, etc.

[0225] The communication interface 910 may include one or more receivers, transmitters, and / or transceivers enabling the computing device 900 to communicate with other computing devices via electronic communication networks (including wired and / or wireless communications). The communication interface 910 may include components and functions for enabling communication over any of a plurality of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or wirelessband), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, the logic unit 920 and / or the communication interface 910 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 902 to one or more GPUs 908 (e.g., memory of one or more GPUs 908).

[0226] I / O port 912 enables computing device 900 to be logically coupled to other devices including I / O component 914, one or more presentation components 918, and / or other components, some of which may be built into (e.g., integrated into) computing device 900. Illustrative I / O component 914 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, etc. I / O component 914 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological input generated by the user. In some cases, input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, pen recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, air gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with the display of computing device 900. Computing device 900 may include depth cameras for gesture detection and recognition, such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof. Additionally, the computing device 900 may include an accelerometer or gyroscope that enables motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the computing device 900 may use the output of the accelerometer or gyroscope to render immersive augmented reality or virtual reality.

[0227] Power supply 916 may include a hardwired power supply, a battery power supply, or a combination thereof. Power supply 916 may provide power to computing device 900 so that the components of computing device 900 can operate.

[0228] The presentation component 918 may include a display (e.g., a monitor, touchscreen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 918 may receive data from other components (e.g., GPU 908, CPU 906, etc.) and output the data (e.g., as images, videos, sounds, etc.).

[0229] Example Data Center

[0230] Figure 10 An example data center 1000 that may be used in at least one embodiment of this disclosure is shown. The data center 1000 may include a data center infrastructure layer 1010, a framework layer 1020, a software layer 1030, and / or an application layer 1040.

[0231] like Figure 10 As shown, the data center infrastructure layer 1010 may include a resource coordinator 1012, grouped computing resources 1014, and node computing resources (“nodes CRs”) 1016(1)-1016(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CRs 1016(1)-1016(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules and / or cooling modules, etc. In some embodiments, one or more node CRs from nodes CRs 1016(1)-1016(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CRs 1016(1)-1016(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more nodes CRs 1016(1)-1016(N) may correspond to virtual machines (VMs).

[0232] In at least one embodiment, the grouped computing resources 1014 may include individual groups of node CRs 1016 housed within one or more racks (not shown), or a plurality of racks housed within a data center in different geographical locations (also not shown). Individual groups of node CRs 1016 within the grouped computing resources 1014 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, a plurality of node CRs 1016, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0233] Resource coordinator 1022 may be configured or otherwise control one or more nodes CRs 1016(1)-1016(N) and / or grouped computing resources 1014. In at least one embodiment, resource coordinator 1022 may include a Software Design Infrastructure (“SDI”) management entity for data center 1000. Resource coordinator 1022 may include hardware, software, or some combination thereof.

[0234] In at least one embodiment, such as Figure 10 As shown, framework layer 1020 may include job scheduler 1028, configuration manager 1034, resource manager 1036, and / or distributed file system 1138. Framework layer 1020 may include a framework for software 1032 supporting software layer 1030 and / or one or more applications 1042 supporting application layer 1040. Software 1032 or application 1042 may respectively contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 1020 may be, but is not limited to, free and open-source software web application frameworks (such as Apache Spark) that can utilize distributed file system 1138 for large-scale data processing (e.g., "big data"). TM(Hereinafter referred to as "Spark") is a type of resource. In at least one embodiment, the job scheduler 1028 may include Spark drivers to facilitate the scheduling of workloads supported by different layers of data center 1000. The configuration manager 1034 may be able to configure different layers, such as the software layer 1030 and the framework layer 1020 (which includes Spark and a distributed file system 1138 for supporting large-scale data processing). The resource manager 1036 may be able to manage computing resources mapped to or allocated to clusters of distributed file system 1138 and job scheduler 1028 or to support clusters of distributed file system 1138 and job scheduler 1028. In at least one embodiment, the clustered or grouped computing resources may include grouped computing resources 1014 in the data center infrastructure layer 1010. The resource manager 1036 may coordinate with the resource coordinator 1012 to manage these mapped or allocated computing resources.

[0235] In at least one embodiment, the software 1032 included in software layer 1030 may include software used in at least a portion of the nodes CRs 1016(1)-1016(N), the grouped computing resources 1014, and / or the distributed file system 1138 of framework layer 1020. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0236] In at least one embodiment, the application 1042 included in the application layer 1040 may include one or more types of applications used at least in part by nodes CRs 1016(1)-1016(N), grouped computing resources 1014, and / or the distributed file system 1138 of the framework layer 1020. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments.

[0237] In at least one embodiment, any of the configuration manager 1034, resource manager 1036, and resource coordinator 1012 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can free the data center operator of data center 1000 from making potentially poor configuration decisions and may prevent underutilization and / or poor performance of the data center.

[0238] According to one or more embodiments described herein, data center 1000 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information. For example, one or more machine learning models can be trained by using the software and / or computing resources described above with respect to data center 1000 to compute weight parameters according to a neural network architecture. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks can be used to infer or predict information using the resources described above with respect to data center 1000 by using weight parameters computed through one or more training techniques (such as, but not limited to, those described herein).

[0239] In at least one embodiment, the data center 1000 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow a user to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0240] Example network environment

[0241] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 9 It is implemented on one or more instances of one or more computing devices 900—for example, each device may include similar components, features, and / or functions of one or more computing devices 900. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 1000, examples of which are described in this document. Figure 10 To describe in more detail.

[0242] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks or one of multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. Where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0243] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server can be implemented on any number of client devices.

[0244] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework supporting software at the software layer and / or application at the application layer. The software or application may respectively include network-based service software or applications. In embodiments, one or more client devices may use the network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a free and open-source software network application framework that can use a distributed file system for large-scale data processing (e.g., "big data").

[0245] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). The core server may assign at least a portion of the functionality to the edge server if the connection to the user (e.g., a client device) is relatively close to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0246] One or more client devices may include the information described in this article. Figure 9At least some of the components, features, and functions of one or more example computing devices 900 described. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming device or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0247] This disclosure can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions, such as program modules, that are executed by a computer or other machine (such as a personal data assistant or other handheld device). Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, and more specialized computing devices. This disclosure can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via a communication network.

[0248] As used herein, the phrase "and / or" relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, "element A, element B, and / or element C" can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, "at least one of element A or element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0249] This document describes in detail the subject matter of this disclosure to meet statutory requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the discloser has envisioned that the claimed subject matter may be embodied in other ways to include steps different from or similar combinations of steps described herein in conjunction with other current or future techniques. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be construed as suggesting any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. One or more processors, including a processing circuitry system, said processing circuitry system being used for: Use one or more machine learning models to identify design data that represents the design of a software product in order to analyze potential security risks associated with the software product; The design data is analyzed using one or more machine learning models to identify the potential security risks associated with the software product; as well as This enables the use of at least one of a display device or a sound device to present a representation of the potential security risks associated with the software product.

2. The processor of claim 1 or more, wherein the design data to be analyzed is identified by: inputting a prompt into a large language model in the one or more machine learning models to identify the design data related to a security threat; and in response to the prompt, obtaining a representation of the design data related to the security threat.

3. The processor of claim 1 or more, wherein the design data to be analyzed is identified by: inputting a prompt into a large language model in the one or more machine learning models to identify the design data related to the security requirements; and in response to the prompt, obtaining a representation of the design data related to the security requirements.

4. The processor of claim 1 or more, wherein the design data to be analyzed is identified by: inputting a prompt into a large language model in the one or more machine learning models to identify the design data associated with a proprietary security mode or an open-source mode; and in response to the prompt, obtaining a representation of the design data associated with the proprietary security mode or the open-source mode.

5. One or more processors as claimed in claim 1, wherein analyzing the design data to identify the potential security risks associated with the software product includes: The large language model in one or more machine learning models is used to determine that the design data failed to mitigate security threats.

6. The processor of claim 1 or more, wherein analyzing the design data to identify the potential security risks associated with the software product includes: The large language model in one or more machine learning models is used to determine that the design data fails to meet security requirements.

7. One or more processors as claimed in claim 1, wherein analyzing the design data to identify the potential security risks associated with the software product includes: The large language model in one or more machine learning models is used to determine whether the design data fails to address proprietary security mode or open source mode.

8. The processor of claim 1 or more, wherein the design data to be analyzed is identified by: inputting a prompt into a large language model in the one or more machine learning models to identify the design data associated with the code snippet; and in response to the prompt, obtaining a representation of the design data associated with the code snippet.

9. The processor of claim 1 or more, wherein analyzing the design data to identify the potential security risks associated with the software product comprises: The large language model in one or more machine learning models is used to determine inconsistencies between the design data and code snippets identified as being related to the design data.

10. The processor of claim 1 or more, wherein the processor is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more multimodal language models; A system that implements one or more large language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; Systems for generating synthetic data using AI; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

11. A system comprising one or more processors, said one or more processors being used to: Use one or more machine learning models to identify code corresponding to a software product in order to analyze potential security risks associated with the software product; The code is analyzed using one or more machine learning models to identify the potential security risks associated with the software product; and This enables the use of at least one of a display device or a sound device to present a representation of the potential security risks associated with the software product.

12. The system of claim 11, wherein the code includes code functions corresponding to the potentially exposed interface.

13. The system of claim 11, wherein the code to be analyzed is identified by: inputting a prompt into a large language model in one or more machine learning models to identify the code corresponding to a potential exposed interface; and in response to the prompt, obtaining a representation of the code corresponding to the potential exposed interface.

14. The system of claim 11, wherein analyzing the code to identify the potential security risks associated with the software product comprises: The large language model in one or more machine learning models is used to determine whether the data flow associated with the code corresponds to a representation of a security risk pattern or whether the code includes a security risk pattern generated based on proprietary or open-source security data.

15. The system of claim 11, further comprising: The one or more machine learning models are used to generate security risk solutions for the potential security risks associated with the software product.

16. The system of claim 11, wherein the system comprises at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more multimodal language models; A system that implements one or more large language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; Systems for generating synthetic data using AI; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

17. A method comprising: Identify design data representing the design of a software product to analyze one or more potential security risks associated with the software product; The design data is provided as at least a portion of the input to one or more machine learning models to identify potential security risks associated with the software product. as well as This enables the use of at least one of a display device or a sound device to present a representation of the potential security risks associated with the software product.

18. The method of claim 17, wherein the input includes a prompt, and the one or more machine learning models include a large language model, and wherein the prompt further includes a representation of threat data, security requirement data, proprietary security data, or open-source data for identifying the potential security risks associated with the software product.

19. The method of claim 17, wherein the input includes a prompt, and the one or more machine learning models include a large language model, and wherein the prompt further includes a representation of code corresponding to the design data for identifying the potential security risks associated with the software product.

20. The method of claim 17, wherein the method is performed by at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more multimodal language models; A system that implements one or more large language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; Systems for generating synthetic data using AI; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

Images