Method and system for assessing conformity of a digital device

Abstract

A computer-implemented method for assessing conformity of an entity (15) with digital elements with a set of requirements (115) is disclosed. This method comprises receiving (S200) a firmware image (102) of the entity (15), identifying (S210) the necessary requirements (115) for the received firmware image (100), and generating (S215) a questionnaire (117) corresponding to the identified necessary requirements (115).

Description

Title: METHOD AND SYSTEM FOR ASSESSING CONFORMITY OF A DIGITAL DEVICERCross-Reference to Related Applications

[0001] NoneField of the Invention

[0002] The field of the invention relates to a computer-implemented method for conformity assessment of security standards of an entity with digital elements (digital device) using an evaluation questionnaire.Background of the Invention

[0003] With the growth of the global Internet of Things (loT) device market, the number of loT devices has increased dramatically with a recent report suggesting that there will be 41 billion loT devices by 2027, up from around 8 billion in 2019 (accessed at https: / / www.businessinsider.com / internet-of-things-report?r=DE&IR=T on 24 Feb. 2020). The purpose of these loT devices is to collect information using embedded sensors or other technologies and to connect and exchange information with other loT devices and processors over the Internet and through a local intranet.

[0004] The manufacturers of so-called “entities with digital elements”, also known as loT devices, often rely on third-party software and hardware components to speed up development processes. In addition to the own code base, these third-party software and hardware components may potentially contain critical security vulnerabilities, and the third- party software and hardware components need to be tested, both for publicly known vulnerabilities and other identified vulnerabilities. The increasing amount of competition in the loT device market means that manufacturers of the loT devices often shorten their release cycles and release those loT devices without extensive security testing. The increasing use of loT devices in private homes, which are connected to the Internet has also become a growing concern because of data privacy and security issues.

[0005] This problem has been recognized and non-governmental organizations, standards setting organizations, industry trade organizations, and governments have released “best practice recommendations,” regulatory frameworks and other legal standards to promote and enforce the practice of releasing well-tested, secure entities with digital elements. There are, however, a number of these recommendations and standards against which the newly released entities need to be certified compliant and this in turn has become an issue due to the large amount of time and resources required to test against the different types of standards.

[0006] The types of standards or regulatory frameworks for which the loT devices need to be made compliant depend on the technology and also the country or region in which the loT device will be used. For example, an loT device used in the United States must be authorized for use by the Federal Communications Commission. Many other countries have similar national or regional requirements that set minimum security standards. These include the following regulatory frameworks:• UK Product Security and Telecommunications Infrastructure Act• Brazilian Cyber Security Requirements for Telecommunications Equipment Act• related regulation from Saudi Arabia, UAE, Oman, China, Taiwan, etc.• loXt labelling scheme• Singapore CLS• Finland NCSC-FI Cybersecurity Label• Japanese loT Security and Safety Framework• German IT Security Label for consumer• US Cyber Trust Mark

[0007] Any loT devices made in one country and exported into another country will often need to be compliant with the standards and requirements of both of the countries. This will require two or more sets of testing against a number of different standards, guidelines, and other best practice recommendations.

[0008] Another challenge is that the regulatory documentation setting out the regulatory frameworks can be difficult to understand and include sections which can be interpreted differently or can be phrased in a vague manner. Furthermore, the understanding of some terms in the regulatory documentation may depend on the language background, technical proficiency or domain knowledge of the person reading the documentation.

[0009] The European Union has proposed a cyber resilience Act (CRA) for improving cybersecurity and cyber resilience in the EU through common cybersecurity standards for entities with digital elements in the EU. The CRA, when introduced, will require manufacturers shall ensure that vulnerabilities of their entity with digital elements are handled effectively for the expected product lifetime or for a period of five years from the placing of the product on the market.

[0010] The regulatory framework for the entity with digital elements is made up of many requirements which differ from country to country. Other requirements concern the technical implementation of the entity.

[0011] Some requirements concern obligations to which publicly available information exists - this includes for example the requirement to publish a time frame how long a device will be supported or publishing a disclosure policy. There are also other requirements or obligations for which public information may or may not exist. For example, manufacturers may be required to conduct a risk assessment, the manufacturers may be required to follow secure software development practices, or the manufacturers may be required to conduct vulnerability management.Prior Art

[0012] A number of patent applications disclose methods for identifying and handling vulnerabilities with digital elements are known. For example, US 2018 / 0121931 Al (assigned to IBM) teaches a method for ensuring compliance of the loT devices in the loT network by providing one or more solutions for the loT devices identified as having performance obligation deficiencies according to a knowledge domain that describes the performance obligations for the plurality of sensor-based devices. The method disclosed in this patent application focusses on checking the compliance of the loT devices regarding performance requirements, but not on compliance with the security requirements of the loT devices.

[0013] US 10,805,165 B2 (Afero) teaches a system and method for managing attributes in the loT network and determining whether the attributes correspond to pre-defined constraints. The system of this patent performs the operations of specifying a plurality of attributes for a corresponding plurality of items of data managed in the loT device and / oran loT service, associating one or more ancillary attributes with one or more of the plurality of attributes wherein the ancillary attributes specify attribute configurations and / or interdependencies between one or more of the plurality of attributes. The ancillary attributes are evaluated to ensure compliance with predefined constraints associated with the plurality of items of data and an indication of compliance is generated if the one or more ancillary attributes are in compliance with the predefined constraints.

[0014] US 10,943,015 B2 (Refirm Labs) teaches a method for continuous monitoring of detecting firmware threats. The patent discloses a system includes a processing pipeline that receives a firmware image from an entity with digital elements, an extractor that receives the firmware image through the processing pipeline, the extractor being configured to determine and extract files within the firmware image, a task queue that receives the extracted files and one or more analysers that: obtain the files from the task queue; and perform at least one type of vulnerability analysis on the files. The system includes a database that stores a log of the at least one type of vulnerability analysis, the log being associated with any of the firmware image and a device identifier of the device.

[0015] US 11,336,697 B2 (Onetrust LLC) teaches a method and system to facilitate collection and management of personal data management documentation requirements and associated data. A master questionnaire is used to solicit information regarding documentation requirements for several contexts in a single interaction and responsive data can be mapped to questionnaires and / or datasets for particular contexts, such as jurisdictions and business sectors. The system can generate graphical user interfaces for presenting the documentation requirement data for a particular context by generating an interface with navigational elements for various contexts, detecting browser state data indicating user manipulation of one or more such elements, and generating a subsequent graphical user interface based on the browser context data. The system configures the subsequent interface to present the requested information in display elements and instructs the browser presenting the subsequent interface to retrieve the requested information using an ontology mapping the requested information to a master dataset.

[0016] US 2021 / 192651 Al (Cambrian Designs, Inc.) teaches a method and system on how to inform data privacy protection systems that utilize personal / corporate privacy policies to engage with digital service providers (DSPs) according to a desired set of protection parameters. The method disclosed addresses both top-down legislative initiatives and thevague corporate Al ethics frameworks that companies are increasingly developing. Currently, prior art documents do not allow to 1) set up their own privacy terms and conditions for engaging with digital service providers; 2) dynamically manage data tracking; 3) start to negotiate terms regarding usage; 4) make recommendations for similar sites with less tracking and to end-users to evolve their privacy charters as they browse; 5) control the flow of data collected and shared with companies outside these services' immediate ecosystems; and 6) build the foundation for an equitable and efficient data marketplace that balances the bargaining power of data creators and buyers.

[0017] The evaluating and mapping in the document are performed by a machine learning algorithm based on tokenizing and classifying content at such sites; and the weighted privacy score is based on user-specific settings for different types of user data in different service categories.

[0018] The document also discloses a compliance reporter which corresponds on behalf of PA users with a DSP site 180 to indicate privacy congruencies, inconsistencies or issues. Recommendations for various site alternatives can be provided preferably to the user along with indications of user-specific privacy violations.

[0019] US 2020 / 412730 Al (EMC IP Holding) teaches a security policy exchange and enforcement for question delegation environments. This document proposes a policy framework for companies and third parties to describe their security requirements for questionnaires in such a way that an intermediate platform can interpret and enforce those requirements. The document discloses a questionnaire submission and response platform where a third party can provide a security policy response to the user’s question and enforcing directives for the accepted security policy. The questionnaire generate is not relevant to an loT device or network.

[0020] US11334063B2 teaches systems and methods for policy automation for a data collection system. A policy automation system for a data collection system in an industrial environment is described. At least one parameter of the system includes a compliance policy, wherein the compliance policy includes at least one of: a data ownership policy, a data use policy or a data format policy. The policies are distributed over a network system to automatically calibrate loT devices.Brief Summary of the Invention

[0021] The invention relates to a computer-implemented method and system for verifying conformity of an entity with digital elements with one or more regulatory frameworks, wherein the regulatory frameworks have a set of requirements.

[0022] The computer-implemented method comprises receiving a firmware image of the entity, identifying the necessary requirements for the received firmware image, and generating a questionnaire with a list of questions corresponding to the identified necessary requirements. The necessary requirements are obtained from regulatory documentation setting out the regulatory frameworks and the questionnaire presents the list of questions for answer by a developer or a manufacturer. The user / manufacturer does not need to review the regulatory documentation but simply answers the questions and receives confirmation whether the entity conforms to the regulatory requirements. This questionnaire can therefore be answered effectively by the developer / manufacturer as the questionnaire only contains the required information.

[0023] The use of a list of questions which cover multiple regulatory standards means that the developer / manufacturer does not need to input the same answers multiple times as they certify whether the entity is compliant with different ones of the standards. The questions can also be designed so that the questions are simple for the developer / manufacturer to understand and use a limited amount of standardised vocabulary which may correspond to different terms used for the same concept in different ones of the regulatory documentation.

[0024] In order to accelerate the answering of the questionnaire, the method further comprise producing a draft set of responses to the questionnaire. The draft set of responses is then checked by the developer / manufacturer, who can then amend individual ones of the responses, or confirm the correctness of the responses. Ones of the draft set of responses can, for example, be generated from previous responses (as noted above) and / or from supplementary information. In a further aspect, the producing of the draft set of response is carried out by an Al system.

[0025] The method of claim further comprises reporting conformity with the requirements for the entity. The requirements comprise at least one of disclosure policies, risk assessments, and the like

[0026] In a further aspect, the method further comprises receiving product documentation and information about the development process. This information is provided by the supplier of the entities and can be updated regularly.

[0027] Should a lack of conformity be identified, then the firmware can be updated in the ones of the entities requiring an update.

[0028] It will be appreciated that the entity is one of a plurality of entities in a network (20).Description of the figures

[0029] Fig. 1 shows a system with a plurality of entities with digital elements.

[0030] Fig. 2 shows a flow diagram for verifying compliance of a firmware image.

[0031] Fig. 3 shows a flow diagram for identifying one or more disclosure policies of a firmware image.Detailed description of the invention

[0032] The invention will now be described on the basis of the drawings. It will be understood that the embodiments and aspects of the invention described herein are only examples and do not limit the protective scope of the claims in any way. The invention is defined by the claims and their equivalents. It will be understood that features of one aspect or embodiment of the invention can be combined with a feature of a different aspect or aspects and / or embodiments of the invention.

[0033] Fig. 1 shows an exemplary system 10 with a plurality of entities 15 with digital elements, such as but not limited to loT devices, arranged in a network 20. The network 20 is an intranet or an extranet and can also be connected to a world-wide network, such as the Internet. The connections between the entities 15 and the world-wide network can be by fixed lines or wireless connections, using for example the Bluetooth, mobile communications, or WLAN protocols.

[0034] The entities 15 include firmware 100. The firmware 100 includes a plurality of chunks of software, such as files, which when run on a processor, control the entities 15.

[0035] A conformity checker 25 is adapted to receive a firmware image 102 from a manufacturer to control the conformity of the firmware 100 in the entity 15 with a set ofrequirements 115. The conformity checker 25 can also access firmware images 102 which have been previously stored in databases. The conformity checker 25 has stored copies of information about the requirements 115, such as copies of disclosure policies 115 for the entity 15 and the firmware 100 in the entity 15.

[0036] One example of the requirements 115 would be for the inclusions of means to reset a password in the firmware 100. The conformity checker 25 would know from the requirements 115 that there was the need for the inclusion of these password resetting means for a particular firmware 100.

[0037] The conformity checker 25 generates a questionnaire 117 corresponding to the set of requirements 115 which need to be fulfilled to meet a standard or regulatory framework. The conformity checker 25 has a list of pre-define questions that can be added to the questionnaire 117. It is also possible to add custom questions to the questionnaire. This questionnaire 117 is displayed to a user, such as the manufacturer of the entity 15 or the developer together with a draft set of responses 118. The draft set of responses 118 can be generated from previously supplied responses or extracted from information stored in local databases. An example of a question would be a question about whether the firmware 100 includes the means to change a password, as noted above. The developer / manufacturer can respond yes or no.

[0038] In one aspect, an artificial intelligence system is used to generate the responses based on the information stored in the local databases or accessed from other sources, as explained later. The artificial intelligence system can employ large language models (LLM), such as OpenAI’s ChatGPT system and natural language processing to generate the responses.

[0039] It has been found that the current 4.0 version of ChatGPT has already been sufficiently trained to produce a set of suitable responses for review and editing by the developer or the manufacturer. It is expected that a local large language model which has been specifically programmed for generating the questionnaire and possible responses is likely to be more effective and indeed will also offer a greater degree of security than using a public, general LLM. Such local LLMs can be programmed using LLM programming systems, for example, Llama, DeepSpeed, Hugging Face Transformers, or LMTuner. These LLM programming are fed with the regulatory documentation of the regulatory regulations and the user-supplied documentation, such as vulnerability management programs, as wellas previously supplied responses and specifications of the entities 15. In a further aspect, the manufacturer or the developer can upload supporting documentation, such as process documentation, product documentation and assessment reports.

[0040] It is possible, however, that the conformity checker 25 does not have the relevant information to be able to produce some of the draft set of responses for the firmware 100 (or some of the chunks of downloaded firmware images 102) and the information cannot be simply uploaded. In this latter case, the conformity checker 25 is tasked to determine draft responses for the set of requirements 115 for the firmware image 102. This can be done in a first step by simply searching for some of the set of requirements 115 using, for example, the vendor’s name and file name. The requirement checker 25 is able to access the worldwide network in order to access vendor websites 120 (or other websites) to access information presented on the website. This access to the vendor websites 120 can be obtained by inputting a uniform resource locator 130 or from other databases.

[0041] The questionnaire 117 is stored as a template in a computer memory and can be accessed by the conformity checker 25. It is intended that the questionnaire 117 provide a simple set of questions which can easily be answered to determine the conformity of the entity 15 with the set of requirements 115. It will be appreciated that there will be different templates for the questionnaire based on the individual set of requirements 115, but that some of the questions in the questionnaire 117 may overlap with each other.

[0042] Fig. 2 shows a flow chart for a computer-implemented method for verifying conformity of the entity 15 with a set of requirements 115. The method comprises receiving, in a receiving step S200, a firmware image 102 of the entity 15 with digital elements. As noted above, the entity 15 with digital elements can be an loT device or other digitally connected device. The firmware image 102 is uploaded to the conformity checker 25 in the receiving step S200 by, for example, the manufacturer or the developer, or can be retrieved from a storage element.

[0043] The set of requirements 115 is identified in an identification step S210 for the firmware 100 of the entity 15. The appropriate set of requirements 115 is identified from the version number of the firmware 100. The questionnaire 117 and a draft set of responses 118 are generated in a questionnaire step S215 by the requirement checker 25 based on the identified set of requirements for the firmware 100.

[0044] The draft set of responses 118 are, as mentioned above, either generated from previously supplied responses or generated from information available either locally or from other websites, or by using the artificial intelligence system. For example, the artificial intelligence system can analyse the user documentation to see whether the ability to change the password on the entity 15 is set out in the set of requirements 115 and user documentation will be analysed to see if functionality for changing the password is set out in the user documentation. The draft set of responses 118 will give the result of this analysis. The manufacturer reviews, in a review step S220, the responses and can either confirm or edit the responses 118 in an edit step S225. Finally, the conformity status with the standard or regulatory framework is made in a conformity step S230 and reported in an output in an output step S235 to the manufacturer or the developer.

[0045] The output can include recommendations on how to make the firmware 100 (or chunks of the firmware 100) compliant. The manufacturer can remedy any deficiencies and re-run the conformity checker 25 on the remedied firmware 100. In one aspect, the remedied firmware can then be uploaded to one or more of the devices 15.

[0046] The results are stored in a storage step and can be passed in a notification step S240 on to a notified body and / or used for a self-assessment.

[0047] One non-limiting example is shown in Fig. 3 which illustrates a computer- implemented method for drafting one example of the requirements. These requirements are disclosure policies 115 for one or more chunks of the firmware images 100. The method set out in Fig. 3 is used if no local copy of the (current) disclosure policy 115 is found. The method comprises a searching step S310 for searching the disclosure policy. This searching step S310 can be carried out in a local database or on the Internet and subsequently the results of the searching step S310 are presented to the developer / manufacturer.

[0048] In some case, the disclosure policies 115 are not found and the searching step S310 comprises an inputting step S312 for inputting a unform resource locator (URL) 130 pointing to a vendor website 120 in the network 20, such as the Internet or a local intranet. The vendor website 120 can be analysed in an analysis step S314, for example by parsing or with the aforementioned artificial intelligence system, to identify vendor information, wherein the vendor information comprises at least one of the disclosure policies 115, vendor contact information 122, confirmation of the public accessibility of the disclosure policy 115, and timelines for status updates 124. The status updates 124 include, for example, intervals inwhich researchers, who disclosed security issues to the vendor under the disclosure policy, will receive updates on the fixing / mitigation process. The results of the searching step S310 are presented to the developer in the questionnaire 117 and the developer can confirm their understanding and the accuracy of the results. The conformity status can be rated in the conformity step S230 and reported in the aforementioned reporting step S235.

[0049] It is possible that the searching step S320 cannot be carried out, for example because no URL can be found, or the website identified by the URL is no longer accessible. Similarly, it is possible for the results returned to be insufficient and that it is not possible to determine the conformity status. In this case, the developer is asked in the questionnaire 117 for the URL to obtain the information.Reference Numerals10 System15 Entity with digital elements, e.g., loT device20 Network25 Compliance Checker30 Extractor100 Firmware102 Firmware image110 Vendor detail115 Requirements117 Questionnaire118 Responses119 Supplementary information120 V endor web site122 Vendor contact information124 Status updates130 Uniform Resource Locator

Claims

Claims1. A computer-implemented method for assessing conformity of an entity (15) with digital elements with a set of requirements (115), comprising: receiving (S200) a firmware image (102) of the entity (15); identifying (S210) the necessary requirements (115) for the received firmware image (100); and generating (S215) a questionnaire (117) corresponding to the identified necessary requirements (115).

2. The method of claim 1, further comprising producing (S215) a draft set of responses (118) to the questionnaire (117).

3. The method of claim 2, wherein ones of the draft set of responses (118) are generated from previous responses or by an artificial intelligence system.

4. The method of claim 2, wherein ones of the draft set of responses (118) are generated from supplementary information (119).

5. The method of claim 1 or 2, further comprising reporting (S235) conformity with the requirements (115) for the entity (15).

6. The method of any of the above claims, wherein the requirements (115) comprise at least one of disclosure policies, risk assessments, and the like.

7. The method of any one of claims 2 to 6, wherein the producing (S215) of the draft set of response (118) is carried out by an Al system.

8. The method of any of the above claims, further comprising receiving product documentation and information about the development process.

9. The method of any one of the above claims, further comprising updating firmware (100) in at least one of the entities (15).

10. The method according to anyone of the above claims, wherein the entity (15) is one of a plurality of entities in a network (20).

Images