Computer-implemented method for managing part reference values in multiple images

Abstract

A computer-implemented method of obtaining part reference values for parts depicted in a set of images. Reuse setting data is received to at least partially control a part reference value for a recurring part that is depicted in more than one image. A first set of part reference values is obtained for a first subset of parts depicted in a first image in the set of images. A second set of part reference values is obtained for a second subset of parts depicted in a second image in the set of images. At least a portion of a part reference value for a recurring part when depicted in the second image is at least partially controlled, in dependence on the reuse data, to accord with at least a corresponding portion of a part reference value for the recurring part when depicted in the first image.

Description

[0001] COMPUTER-IMPLEMENTED METHOD FOR MANAGING PART REFERENCE VALUES IN MULTIPLE IMAGES

[0002] Technical Field

[0003] The present disclosure relates to computer-implemented methods and systems for managing reference values associated with parts depicted across multiple images.

[0004]

[0005] Many types of electronic document contain images. Images containing diagrams, charts, etc., are often accompanied by descriptive text in a variety of electronic documents, including patent applications, academic papers, and technical manuals. This descriptive text aids in the understanding of the images, often referred to as “figures” or “drawings”, by providing additional context or detailing the elements present within the images. The process of generating this text, however, is typically manual and can be time-consuming, especially when dealing with a large number of complex images.

[0006] Generative artificial intelligence (Al) offers the ability to generate text based on natural language prompts, and the availability of neural network large language models such as ChatGPT and LLaMa has resulted in widespread interest in how generative Al can be used commercially.

[0007] A proposed use for generative Al is to assist the preparation of patent specifications. More particularly, attempts have been made to prepare the textual description part of a patent specification based on a prompt derived from a set of patent claims, with varying degrees of success. One particular challenge faced in preparing a patent specification, or indeed similar types of electronic documents, using generative Al is in preparing not just the text but the accompanying images of the patent specification.

[0008] Technical documentation uses part reference values to label and track elements within drawings, diagrams, and figures. These identifiers enable communication between different sections of documentation and allow identification of components across multiple views or representations. In fields where documentation includes multiple related images to represent different aspects or views of components,maintaining part reference values across the documentation becomes a practical consideration. Furthermore, different organisations use different conventions.

[0009] Manual methods for tracking part reference values involve individual assignment and verification processes. Users maintain records of assigned identifiers and perform checks when elements appear in multiple locations. This approach requires verification of part reference values assignments, number sequences, and confirmation of consistency across different representations of components in the documentation.

[0010] Electronic documentation systems face various technical challenges in part reference value management. These challenges relate to the handling of part reference values across image sets. The relationship management between different representations of components in these images can present difficulties. Documentation systems encounter technical obstacles when managing identifiers across multiple images.

[0011] The management of part reference values affects documentation workflow processes. The verification and updating procedures require significant time resources. Documentation quality depends on the accurate assignment and maintenance of part reference values. The process of updating documentation introduces technical considerations regarding the maintenance of part reference values throughout the documentation system.

[0012] In examples, there is provided a computer-implemented method of obtaining part reference values for parts depicted in a set of images. Reuse setting data is received to at least partially control a part reference value for a recurring part that is depicted in more than one image. A first set of part reference values is obtained for a first subset of parts depicted in a first image in the set of images. A second set of part reference values is obtained for a second subset of parts depicted in a second image in the set of images. At least a portion of a part reference value for a recurring part when depicted in the second image is at least partially controlled, in dependence on the reuse setting data, to accord with at least a corresponding portion of a part reference value for the recurring part when depicted in the first image.The control of part reference values for recurring parts may include a default reuse control scheme, to be applied by default across the set of images or an image document including the set of images. In addition, the control of part reference values for recurring parts may include an element-level reuse control setting. Element-level reuse control settings may be set at a group-of-images level, an image level, a group-of-parts level and / or at a part level. A default reuse control scheme may involve the reuse of part reference values by default or may involve a numbering scheme which does not reuse part reference values by default. In a case in which the default reuse control scheme reuses part reference values by default, the element-level reuse control setting may be used to override this so as not to reuse part reference values for one or more identified groups-of-images, images, groups-of-parts and / or parts. Conversely, if the default reuse control scheme does not reuse part reference values by default, the element-level reuse control setting may be used to override this so as to reuse part reference values for one or more identified groups-of-images, images, groups-of-parts and / or parts.

[0013] The reuse setting data may include data defining one or more reuse settings. These may include a user’s default settings, a project or document level setting, and / or element-level control settings. The reuse setting data may include a combination of these settings. The use of an element-level record above the group-of-parts level and / or at a part level, i.e. at a group-of-images level, an image level is advantageous because the user can control reuse settings for relatively large numbers of parts without needing to set, and maintain these at a group-of-parts level and / or at an individual part level.

[0014] One or more of the reuse settings may have a primary or sole purpose of controlling the reuse of part reference values. Alternatively, or in addition, one or more of the reuse settings may have more than one purpose, including controlling the reuse of part reference values and at least one other purpose. For example, a reuse setting may take the form of a numbering mode setting. Such a numbering mode setting may have both a purpose of controlling a reuse of a portion, for example a lower-order portion, of part reference values as well as a purpose of controlling a scheme wherebya different portion, for example a higher-order portion, of part reference values is generated.

[0015] Image relationship data may be set by the user that defines relationships between images and identifies related image subsets. The reuse settings may include such image relationship data. Part reference values can, for example, be controlled based on relationship data indicating a relationship between the first and second images.

[0016] The method may include formulating prompt data for a neural network large language model to generate description text for the second image. This involves deriving part input data for parts depicted in the second image and generating corresponding prompt data. The prompt data is transmitted to the neural network large language model, which returns description text data for the image.

[0017] Image relationship input data may be derived from the image relationship data and included in the prompt data. A common graphical user interface component used to specify relationships between images can be used to control both relationship input data items and at least some of the reuse settings for the the part reference values, providing a user with intuitive and unified control of both the image relationship input data and the reuse settings.

[0018] The method may provide interactive preview and selection of part reference values. A candidate part reference value can be generated for a recurring part when it appears in the second image, controlled to correspond to its value from the first image. This candidate value is displayed in a preview that allows user selection. User input can then add a callout to the second image that associates the selected reference value with the recurring part.

[0019] For parts that have not yet been depicted in any image, the method may generate candidate reference values. These values may be controlled according to predetermined increments from the last-used reference value in a series, or for recurring parts on the basis of reuse setting data. These candidate values can be previewed and selected through user input before being associated with the newly depicted or recurring part via a callout in the second image.

[0020] The second subset of parts may be determined through an interactive process in an image editor. Users can iteratively add callouts to the image, associate them with specific parts, and populate the second subset accordingly. The system can display partreference values while preventing modification of recurring part references but allowing changes to newly depicted part references.

[0021] The method is particularly applicable to patent specification figures, where the part reference values are used consistently across both the figures and their corresponding written descriptions in the specification.

[0022] The method may include element-level reuse control settings that determine whether specific portions of part reference values, particularly modulo remainder portions, should be reused when the same part appears in later images. These control settings can be applied at one or more different levels - to specific part reference values, groups of parts, individual images and / or to groups of images within the image set.

[0023] Part reference values may follow a structured format expressed as 100 x H + R, where H represents a high-order portion and R is an integer between 0 and 99 serving as the modulo remainder portion. In certain numbering configurations, the reuse setting data can specify whether both the high-order portion and modulo remainder portion should be reused for recurring parts across images.

[0024] The method may implement various numbering mode settings that can be configured at different levels of granularity - for individual images, subsets of images, or globally across all images. These settings can include specific high-order numbering modes that control how the high-order portion of part reference values is managed.

[0025] The method may generate high-order number portions of part reference values based on image indices associated with each image in the set. This creates a systematic relationship between part references and their location within the document structure.

[0026] Two distinct high-order numbering modes may be implemented: a first-appearance mode and an image-indexed mode. In first-appearance mode, a recurring part's high-order number portion remains fixed based on the lowest image index where it first appears. In image-indexed mode, the high-order portion changes dynamically based on the current image index where the part appears.

[0027] An incremental high-order numbering mode may also be provided, where the high-order portion of part reference values progresses according to a predetermined increment, independent of image indices. In this mode, reference values can remain fixed when a recurring part appears within a defined subset of images, only incrementing when the part appears in images outside that subset.The method may apply a reference value generation setting, for example a high-order numbering setting, globally across all images while maintaining multiple element-level reuse control records. Whilst element-level records may be maintained at the part level or group-of-parts level, a more powerful control setting can be achieved at the image level or group-of-images level. Each such record can for example define different subsets of images with specific part reference value handling rules, allowing control over how part references are controlled as sets of parts associated with images or groups of images. By associating these with images or groups of images, the groups do not need to be separately maintained where a part-figure mapping is already maintained, both for the images that for which part reference values are currently being controlled and for those from which the part reference values are derived. Element-level reuse control records specifying an image subset may be applied at an image level, such that individual images may have different subsets of related images, thereby providing a relatively high granularity of control of the reuse of part reference values without requiring the user to specify the reuse of part reference values at the individual part level. Alternatively, or in addition, element-level reuse control records specifying an image subset may be applied at a group-of-images level, again providing a relatively high granularity of control of the reuse of part reference values without requiring the user to specify the reuse of part reference values at the individual part level. Elementlevel reuse control records specifying an image subset at an image level provides greater flexibility of reuse than element-level reuse control records specifying an image subset applied at a group-of-images level.

[0028] The method may associate individual element-level reuse control records with specific images, each record defining which subset of images should contribute to part reference value reuse for recurring parts in that image. This enables precise control over how reference values are inherited on an image-by-image basis.

[0029] Part reference values may be stored in association with a parts list, with automatic updates occurring when reuse setting data changes. A maintenance function can automatically update part reference values in response to changes in the image set, ensuring consistent referencing across all affected images.

[0030] The method may include safeguards to maintain reference integrity. When attempting to delete images, the system can prevent deletion of parent images that areused to define part reference values in child images. Similarly, when reordering images, the system can prevent child images from being placed before their parent images in the sequence, maintaining the logical flow of reference inheritance and preventing circular dependencies.

[0031] These maintenance functions help ensure that part reference values remain consistent and valid throughout the document, even as the image set undergoes modifications or reorganization.

[0032] The method may include a manual override feature through reuse setting data that allows users to specify custom part reference values for recurring parts, overriding the automatically generated values. This flexibility accommodates special cases for individual parts while maintaining automated processing for other parts.

[0033] When parts depicted in an image are reordered, the method can automatically recalculate part reference values for recurring parts based on the current reuse setting data. This ensures that reference consistency is maintained even when the document structure is modified through reordering.

[0034] Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.

[0035] Brief Description of the Drawings

[0036] Figure 1 is a schematic block diagram illustrating the main components of a document generation system;

[0037] Figure 2 is a schematic block diagram showing the main components of a front end of the document generation system of Figure 1;

[0038] Figure 3 is a flow chart showing the main operations performed by a platform forming part of the document generation system according to claim 1;

[0039] Figure 4 shows an illustration of an example of a first web app window presented to a user of the document generation system of Figure 1;

[0040] Figure 5 shows an illustration of the first web app window of Figure 4 following text input by the user;Figure 6 is a flowchart showing the main operations performed by a natural language processing pipeline implemented by the document generation system of document DI;

[0041] Figure 7 shows an illustration of an example of a second web app window presented to a user of the document generation system of Figure 1;

[0042] Figure 8 shows another illustration of the second web app window of Figure 7 following editing of automatically-generated parts in a parts list by the user;

[0043] Figure 9 shows a further illustration of the second web app window of Figure 7 following entry of additional parts to the parts list;

[0044] Figures 10 and 25 show illustrations of an example of a third web app window presented to a user of the document generation system of Figure 1;

[0045] Figure 11 is a flow chart showing the main operations performed in a process to identify parts included in an image;

[0046] Figures 12 and 26 show illustrations of an example of a fourth web app window presented to a user of the document generation system of Figure 1; and

[0047] Figure 13 is a flow chart showing the main operations performed in an alternative process to identify parts included in an image;

[0048] Figures 14 to 24 show flow diagrams illustrating a process for managing part reference values across multiple images.

[0049] Detailed Description

[0050] The present disclosure relates to computer-implemented methods and systems for managing reference values associated with parts depicted across multiple images. A part reference value may consist of a numeric string such as “100” or it may consist of an alphanumeric string including an alphabetic portion and a numeric string, for example “100 A” or “SI 00”. A value reuse may be reuse in full or only partial reuse, for example a part reference value of “120” may be used in one image and the reuse may take the form of reusing the modulo 100 remainder (i.e. “20”) in a following image, for example taking a value of “220” or “420”. Alternative value reuse may be reuse in full of only a numerical portion, for example a part reference value of “120A” may be used in one image and the reuse may take the form of reusing the numeric portion andvarying the alphabetic portion, e.g. incrementing the alphabetical portion using an alphabetical increment, in this example to “120B”.

[0051] According to examples, the methods and systems for managing reference values associated with parts is included in computer-implemented methods and systems of generating text associated with the set of images image. Such a method may comprise, or a system may implement, receiving unstructured text data describing one or more entities at least one of which is associated with the image and processing the unstructured text data to generate corresponding structured text data. A set of parts can then be derived using the structured text data, optionally supplemented by additional parts added by the user manually, and a subset of that set of parts determined that identifies parts that are present in the image. Prompt data can then be formulated using the received unstructured text data and the subset of the set of parts. The prompt data is then sent to a neural network large language model, resulting in description text corresponding to the image being subsequently received from the neural network large language model. Identifying parts that are present in the image and incorporating data identifying those parts in the prompt data can provide for more accurate description data for the image.

[0052] In examples, the user-input unstructured text data is processed to identify noun phrases in the text and each new instance of a noun phrase is assigned as a part in the set of parts. One way of identifying the noun phrases comprises tokenising the unstructured text data and for each token, labelling the token with a corresponding part of speech to generated labelled text data, which is then parsed to determine dependency information. Core nouns can then be identified from the labelled text data using the dependency information, and the noun phrases then identified using the identified core nouns. To avoid the same part being listed more than once, the identified noun phrases may be co-referenced, for example using a set of heuristic rules, to identify matching noun phrases, and then the same label may be applied to matching noun phrases.

[0053] The subset of the set of parts for a respective image can be determined either based on user input or automatically or a combination of user input and automatically. In examples, the subset of parts is determined by displaying the image to a user in an image editor and receiving user input adding callouts to the image and associating the added callout to a part in the set of parts, wherein the subset of the set of parts ispopulated with the parts associated with the callouts. In another example, image data corresponding to the image may be analysed with image analysis software to identify automatically features within the image likely to correspond to parts in the set of parts, and the set of parts is populated with the parts that are likely to correspond to the identified features. The subset of parts may subsequently be displayed to a user for amendment based on user input. In such examples, a reference numeral may be assigned to the added callout either based on user input or manually and the prompt data may be formulated using the reference numerals linked to the subset of the set of parts to enable the references to be incorporated in the text data received from the neural network large language model.

[0054] In examples, the derived subset of parts is presented to a user and, optionally, manually edited for each image prior to the generation of the prompt data. In this way, an accurate respective subset of parts depicted in each image can be produced.

[0055] In examples, the received unstructured text data is a set of patent claims, and the image is a figure for a patent specification such that the text data received from the neural network large language model provides a description for the image. By generating description for a set of images, a detailed description section for a patent specification can be developed. In an example, the detailed description for the set of images is generated iteratively image by image so that each image is generated based on prompt data that is specific to that image.

[0056] Unstructured text data, such as user-generated or edited patent claims, sentences or paragraphs describing one or more entities associated with an image, may lack a predefined model or format, making it difficult to systematically extract relevant information. Thus, the process of converting unstructured text data into a structured format suitable for generating descriptive text is advantageous. Additionally, the task of describing, or indeed initially identifying, parts of the image that need to be described is a complex and often error-prone process. Whilst final text and the images are, in typical patent specifications, highly related, not all parts that are described in the input data, such as the claims, may be present in the images, and vice versa.

[0057] In examples, there is provided a computer-implemented method of generating text associated with a system, process and / or apparatus to be shown in a plurality of images including a first image and a second image. The method comprises receivingfirst text data describing one or more aspects of the system, process and / or apparatus shown in the first image, receiving first user-input part subset data for determining a first subset of a set of parts to be shown in the plurality of images, the first subset identifying parts that are present in the first image, and formulating first prompt data for a neural network large language model to generate first description text corresponding to the first image. The formulating of the first prompt data comprises deriving first description data associated with the first image from the first text data, deriving first part data associated with the first image from the first user-input part subset data, and generating first prompt data comprising the first description data and the first part data. The first prompt data is sent to the neural network large language model and in return the first description text data is received for the first image from the neural network large language model. The method also comprises receiving second text data describing one or more aspects of the system, process and / or apparatus shown in the second image, receiving second user-input part subset data for determining a second subset of the set of parts to be shown in the plurality of images, the second subset identifying parts that are present in the second image, and formulating second prompt data for the neural network large language model to generate second description text corresponding to the second image. The formulating of the second prompt data comprises deriving second description data associated with the second image from the second text data, deriving second part data associated with the second image from the second user-input part subset data, and generating second prompt data comprising the second description data and the second part data. The second prompt data is sent to the neural network large language model and second description text data for the second image is received in return from the neural network large language model. The first user-input part subset data and the second input part subset data may be received during an image editing process in which the first image and the second image are edited.

[0058] In examples, there is provided a computer-implemented method of generating text associated with an image, comprising receiving user-input text data describing one or more entities, identifying one or more entities that are associated with the image, and formulating prompt data for a neural network large language model to generate description text corresponding to the image. The formulating of the prompt data comprises deriving description data from the user-input text data, deriving part datacorresponding to the identified one or more entities, and generating prompt data comprising the description data and the part data. The prompt data is sent to the neural network large language model and description text data for the image is received from the neural network large language model.

[0059] System Architecture

[0060] As shown in Figure 1, a document generation system includes a platform 1, a natural language processing (NLP) processing system 3 and a neural network large language model (LLM) system 5.

[0061] The platform 1 includes a front end 7 that allows a user to interact with the document generation system by entering user input 13 and accessing a document that is generated based on that user input. As shown in Figure 2, in this example the front end 7 includes a web server 41, a text editor 43, an image editor 45 and a numbering engine 47. Note that the terms “numbering” and “numerals” herein is used in a general sense to refer to the values used in part reference values rather than strictly in relation to numeric strings only. The numbering engine is a computer software module or set of modules configured, when executed, to perform the methods of control, generation and management of part reference values to be described in relation to Figures 10 to 26 below. The web server 41 enables the user to interact with the document generation system over the Internet using a web application (referred to herein for brevity as a “web app”), or set of web apps, running on conventional web browser software installed on a client computer such as a desktop or laptop computer. As will be discussed in more detail hereafter, the web server 41 can embed the text editor 43 and / or the image editor 45 and / or the numbering engine in downloadable resources in a web app or apps, which provide various web browser-based application windows to enable the user to enter input data 13 in the form of text and to enter one or more images using a drawings editor and / or by importing image files. Based on the text entered input data 13, the document generation system generates text describing the one or more entered images.

[0062] Returning to Figure 1, the platform 1 also includes a database 9 and a worker process 11. The database 9 includes a record for each interaction by a user to generate a document, while the worker process 11 images the processing of each interaction using the record stored in the database 9 for that interaction. Database records aremaintained for the interactions because the interactions between the platform 1 and the NLP processing system 3 and the neural network LLM 5 can involve significant time delays and during that time delay the platform 1 stores data in the corresponding record in the database 9 and frees up processor resources for other user interactions.

[0063] The NLP processing system 3 includes an interface 15 implementing an NLP processing API, an NLP pipeline 17, NLP model storage 19 and a serialiser 21. In this example, the NLP processing API interface 15 is configured to receive unstructured text data for analysis. The NLP pipeline 17 processes the unstructured text data to output a structured text data model, which is stored in the NLP model storage 17. The NLP pipeline includes NLP functionality available in spaCy™, an advanced NLP library available at https : / / github.com / expl osion / spaCy. As will be described in more detail hereafter, in this example the NLP pipeline 17 executes routines from the spaCy library together with heuristic rules to generate the structured text data model. As the NLP processing system 3 is remote from the platform 1 in this example, the structured text data model is serialised by a serialiser 21 for transmission back to the platform 1. In this example, the serialiser encodes the structured text data model as a JSON object for transmission.

[0064] The neural network LLM system 5 includes an interface 23 implementing an LLM API and an LLM 25. The interface 23 is configured to receive unstructured text data, hereafter referred to as prompt data, which is input to the LLM 25. In this example, the LLM 25 is the generative pre-trained transformer model called GPT-4™, or a variant thereof, provided by OpenAI to generate an output based on input prompt data. In this way, the prompt data can be used to ask the LLM 25 to generate a desired text output. The extent to which the subsequently generated text output matches the desired text output is dependent on the prompt data. Tailoring the prompt data to improve the quality of the generated text output in comparison with a desired text output is commonly referred to as prompt engineering.

[0065] Document Generation

[0066] The operation of the document generation system to generate a document descriptive of an image will now be described with reference to Figures 3 to 12. To aid understanding, in tandem with describing generically the operations performed, aspecific user interaction will be described, by way of example only, in which a description of a patent figure is automatically generated in response to prompt data that has been engineered using a set of patent claims.

[0067] As shown in Figure 3, after a user has navigated to the web server 41 of the platform 1, the user interaction begins with the platform receiving, at block SI, unstructured text data from the user. More particularly, the user navigates to a first web app window 51 as illustrated in Figure 4. The first web app window 51 provides three ways in which a user can enter text data. The first way is a text editing region 53 for the text editor 43 which enables the user to type in text data. The second way is an upload button 55 that enables the user to search a directory structure to identify and upload a text file. The third way is a file uploader box 57 that enables the user to drag and drop a text file into the file uploader box 57 to upload the text file. The first web app window also includes a process text button 59.

[0068] Figure 5 shows the first web app window 51 following text entry, with the entered text displayed in the text editing region 53. It will be seen that the entered text of the specific example is a set of patent claims formed by an independent claim and four dependent claims. When the text has been entered, the user activates the process text button 59, which initiates the front end 7 supplying unstructured text data corresponding to the entered text to the database 9 for processing by the worker process 11.

[0069] Returning to Figure 3, the worker process 11 generates, at block S3, a record in the database 9 for the unstructured text data and associates the record with a record identifier. The worker process 11 then sends, at block S5, the unstructured text data to the interface 15 of the NLP processing system 3 for processing by the NLP pipeline 17. The operation of the NLP pipeline 17 in this example will now be described with reference to Figure 6.

[0070] Following receipt, at block S21, of the unstructured text data, the NLP pipeline 17 tokenises, at block S23, the unstructured text data using a tokenizer forming part of the spaCy library. This splits the text into tokens, with each token corresponding to a word or a punctuation mark. The NLP pipeline 17 then tags, at block S25, the tokenized text data with labels indicating parts of speech, using a tagger that is also part of the spaCy library. The NLP pipeline 17 then parses, at block S27, the labelled text data togenerate dependency parse information, which indicates grammatical relationships between words represented by the tokens, using a parser that forms part of the spaCy library.

[0071] In this example, the NLP pipeline 17 then identifies, at block S29, units of measurement and assigns single tokens to each unit of measurement. The NLP pipeline 17 then applies, at block S31, a set of heuristic rules to identify noun phrases and assigns a token to each noun phrase. An identified noun phrase may consist of a single word or multiple words. The set of heuristic rules identifies the noun phrases using a combination of the labelling and dependency information to identify core nouns, and then rules identifying when a word dependent on a core noun is actually part of a noun phrase incorporating the core noun. For example, there may be a rule that states that if the core noun “server” is immediately preceded by a dependent noun “web”, then there is actually a noun phrase “web server”.

[0072] At this stage of the processing, a noun phrase corresponding to a particular entity may appear multiple times in the unstructured text data, and each noun phrase will have a different token for each appearance. The NLP pipeline 17 performs, at block S33, coreferencing to identify matching noun phrases and establishes a graph relationship between the noun phrases. The NLP pipeline then generates, at block S35, structured text data corresponding to the received unstructured text data.

[0073] The structured text data output by the NLP pipeline 17 is stored in the NLP model storage 19, serialized by the serializer 21 to generate a corresponding JSON object, and then transmitted back to the platform 1 as a data package including the record identifier.

[0074] Returning again to Figure 3, the platform 1 receives, at block S7, the data package conveying the structured data and the record identifier and saves the structured data in the record in the database 9 identified by the database identifier. The worker process 11 then generates, at block S9, a set of parts for the entered text with each part of the set of parts corresponding to a noun phrase, and displays the set of parts to the user in a second web app window.

[0075] Figure 7, 8 and 9 show illustrations of the second web app window 71 for the specific example at different stages of user interaction. As shown in Figure 7, in the main region 73 of the second web app window 71 the originally entered text has beenformatted as a list of features with the noun phrases identified by the NLP pipeline 17, hereafter referred to as candidate noun phrase, underlined. In a sub-region 75 of the second web app window 71 is a scrollable list of parts that has been populated with the candidate noun phrases. At the bottom of the scrollable list is a “add new” button 77 (not shown in Figure 7 but visible in Figure 9). The second web app window 71 also includes a “next step” button 79.

[0076] It is apparent from a review of the candidate noun phrases underlined in the main region 73 of the second web app window 71 that the NLP pipeline 71 has mischaracterised some of the candidate noun phrases. For example, words such as “entry” have been suggested as candidate noun phrases. To address this eventuality, the second web app window 71 allows the list of parts in the sub-region 75 of the second web app window to be edited. In particular one or more of the following types of editing functionality may be provided:

[0077] • each part may be displayed in association with a trash bin symbol and a part can be removed from the list of parts by activating the associated trash bin symbol;

[0078] • the name of the part can be edited in the sub-region 75 of the second web app window; and

[0079] • a part can be added to the list of parts by clicking the “add new” button and then adding the new part.

[0080] Figure 8 shows an illustration of the second web app window 71 for the specific example after parts have been removed from the list of parts and the names of some parts have been edited. Figure 9 shows the list of parts after new parts have been added to the list of parts. The user may want to add parts to the list of parts to include parts that are not disclosed in the originally entered text but which do appear in an image. Once the user has completed editing the list of parts, the user activates the “next step” button.

[0081] Returning to Figure 3, following the receipt, at block SI 1, of edits to the set of parts by the user as discussed above, the platform 1 receives, at block S13, image data for an image. The platform 1 then determines, at block SI 5, a subset of the set of parts corresponding to parts present in the image.In this example, the image data is generated and the subset of parts is determined using a third web app window 81, illustrated in Figure 10, in which the image editor 45 is embedded. As shown in Figure 10, the image editor is accessed via a main region 83 of the third web app window 81. The image editor includes user controls 85 that enable a user to generate a drawing and / or import image file(s) and / or a combination of both, all of of which are generally understood to be “images” within the meaning of the present description. The image editor is in examples based on a modified version of the Excalidraw™ in-browser drawing software, which is available at https: / / github.com / excalidraw / . The modifications include a callout function which allows a user to label a part in a drawing, or in a pasted-in image, and to assign the part an associated reference numeral.

[0082] To one side of the main region 83 of the third web app window 81 is displayed the list of parts in two subsets. A first subset, displayed in a first sub-region 87, represents an ordered subset of the parts list for parts that are depicted in the currently displayed image while the second subset, displayed in a second sub-region 88, lists parts in the set of parts that have not (yet) been indicated to be depicted in the currently displayed image. A part can be moved from the second subset to the first subset by selecting the part in the second subset, which causes a callout to be generated that can be dragged onto the image by the user and attached to the corresponding depiction of the part in the image. By use of the numbering engine 47, a part is allocated an initial candidate reference value, and this candidate number is displayed as a preview when in the second sub-region 88, and the initial candidate reference value remains allocated, or is copied over to, the first subset when it is moved from the second subset to the first subset. The initial candidate parts displayed as a preview can be one, or both, of two kinds: fixed part references, that do not change (at least for the drawing and labelling of current image) as parts are allocated to the image using callouts, and dynamic part references, that do change (again, at least for the drawing and labelling of current image) as more parts are allocated to the image using callouts. Fixed candidate part references result from reservation of part references for parts that have been depicted in a different image, and if depicted in the current image will recur. Dynamic part references are for parts that have not been indicated as depicted in any image, or at least in a current group of images. The dynamic part references are incremented each timea part is allocated to the current image using a predetermined increment (for example an increment of 1, 2 or 5 numbers) with the initial number having a predetermined lowest order digit (0, odd or even or 5). Each of these may be predetermined by user specified defaults. In the example shown in Figure 10, various parts have been allocated to the image and given the part reference values 102, 104, ... 114 that are shown in first sub-region 87. Remaining parts have not yet been allocated to the image and have each been given the candidate part reference value 116 that is shown in the second sub-region 88. If any one of these remaining parts is allocated to the image by adding a callout, the relevant candidate part reference value, i.e. 116, is added to the callout in the image and associated with the part as shown in the image, and the relevant part is moved to the first sub-region 87. Furthermore, the dynamic candidate part reference value for the remaining parts is updated to 118 (the predetermined increment being 2 in this example.)

[0083] In an alternative, parts in the second subset may have no initial candidate reference values displayed and part reference values may be generated, or allocated, and displayed only after the callout is created and associated with a part in the second subset. In any case, the method automatically causes the part to be assigned a reference value and to be moved from the second subset to the first subset.

[0084] A third sub-region 89 is displayed to the side of the main region 83 of the third web app window opposing the first and second sub-region 87 and 88. A text box 91 is displayed in the second sub-region 89 and enables the user to add text describing the image. Further text boxes may be displayed in the second sub-region 89 to enable the user to add text describing particular parts in the image, for example information can be added explaining how a parts that is present in the originally entered text interacts with parts that are present in the originally entered text. A figure relationship GUI element 92 is displayed in the second sub-region 89, allowing a user to specify one or more related images, being parent images, for the current image. In the example shown in Figure 10, the image being generated is “Figure 1” and there are no lower-ordered images and therefore there no related images to select.

[0085] Figure 11 is a flow chart summarising an example of operations performed in this example to move a part from the set of parts into the first subset listing parts that are present in the image. The image is displayed, at block S41, to the user and userinput is received, at block S43, adding a callout to the image and associating the callout with a displayed part. Further user input is received, at block S45, associating the callout and a part in the set of parts. A reference numeral is then associated, at block S47, with the callout and the corresponding part is moved from the second subset of the set of parts to the first subset of the set of parts.

[0086] Returning to Figure 3, once editing of the image has been completed and all parts in the image have been added to the first subset for the set of parts, prompt data may be formulated, at block S17, for the generation of text corresponding to the image. This prompt data may include:

[0087] • those features in the originally entered text, as shown in the main region 73 of the second web app window 71, that included noun phrases corresponding to parts in the first subset of the set of parts;

[0088] • the first subset of the set of parts including their associated reference numerals; and

[0089] • any additional text data entered by the user in the third web app window 81 describing the image or parts of the image.

[0090] The platform 1 then sends, at block SI 9, the prompt data to the neural network LLM system 5 in the form of one or more prompts. Subsequently the platform receives, at block S21, the description data for the image from the neural network LLM system 5 and displays, at block S23, the description data to the user.

[0091] Figure 12 shows a fourth web app window 101 showing text generated for the image of the specific example (“Figure 1”) as illustrated in Figure 11. The generated text may then be edited by the user to correct any errors introduced by the neural network LLM system 5.

[0092] Controlling part reference values across a set of images

[0093] Next, obtaining part reference values for parts depicted in a set of images will be described. Where the subject of the text generation is a patent specification, the set of images may be the whole set of images for the patent specification, or a subset of the images. For example, one or more images may be added to represent the prior art and these may use a different part numbering scheme than the examples of the subject-matter of the claims. In that case, the set of images across which part reference values are controlled in accordance with examples may be a group of images within the entire set of images. And further there may be related subsets of images within that main subset, the purpose of which is not just to define textual description input but also to, optionally control the reuse of at least a lower-order portion of a previously-used part reference value, as will be apparent from the following description of examples.

[0094] Figure 14 illustrates a flow diagram showing a process for managing part reference values across multiple images. The process includes receiving reuse setting data S61 for controlling part reference values, obtaining part reference values for parts depicted in first and second images S63, S65, controlling part reference values for recurring parts S67, and obtaining recurring part reference values based on reuse setting data S69. The flow diagram depicts how part reference values can be managed when parts appear across different images while maintaining consistency through the use of reuse setting data.

[0095] The process may involve identifying at least one part that is in the first image and recurs in a second image in the set of images, and for that at least one part, allocating a part reference value in a second set of part reference values for a second subset of parts in the second image. The same process may be applied in relation to further recurrences in other images, for example third and fourth images in the set of images. At least a portion of the part reference value is then automatically reused, i.e. programmatically copied or reallocated, from a corresponding part reference value in the first set of part reference values. The recurrence of a part may be identified based on the same part identifier being allocated to the respective images, for example by the user drawing callouts on the images and allocating the callouts to parts, for example by selecting the part name associated with the part identifier from a list of part names.

[0096] The reuse setting data received at block S61 may include one or more of numbering scheme parameters, hundreds digit allocation mode parameters, document identifiers, creation and update timestamps, and figure relationship settings that define number reuse between parent and child figures. Additional numbering control data encompasses manual override settings for part numbers, part sorting order, figure mapping data, and figure locking status information. These parameters influence the processing of recurring parts in subsequent images and establish parameters that affectoperations performed by blocks S63, S65, and S67. Parts also have associated PartID, whereby the system uniquely identifies part, and part names which are used in the text when associated with a corresponding depiction of a part in an image using a callout in the image, as will be described in more detail below.

[0097] Block S63 obtains a first set of part reference values for parts depicted in a first image, processing various types of part data including PartID, PartName, optional descriptions, reserved modulo remainder values, first appearance figure information, candidate numbers, computed part numbers, and sort order values. The block may access additional data such as part-figure mapping data, document settings parameters, and figure relationships data for figure subsets. Block S63 operates in conjunction with subsequent blocks to process data for potentially recurring parts across different images, and can handle part-figure mapping data in both FirstAppearance and Incremental modes, retrieving computed part number values unless a manual override is active.

[0098] Block S65 obtains a second set of part reference values from a second image in the set of images, operating sequentially after blocks S61 and S63, and outputs processed data to block S67. The block processes part-figure mapping data to determine associations between parts and figures while accessing parts entities and part-figure mapping to retrieve reference information. Block S65 processes data according to document settings parameters, including reuse numbers flag, manual overrides, first appearance figure field, computed part number values, hundreds digit allocation mode, and numbering scheme, working in conjunction with block S63 to gather reference data values from different images that may include recurring parts from the first image.

[0099] Block S67 receives input from block S65 and provides output to block S69 in the sequential process flow, controlling part reference values for recurring parts appearing in the second image based on reuse setting data received from block S61. The block utilizes part-figure mapping to track and process parts appearing across multiple figures, referencing FirstAppearanceFigure field and processing CandidateNumber values for parts appearing in multiple figures. Block S69 functions as the final processing stage, obtaining reference values for recurring parts while referencing DocumentSettings parameters including NumberingScheme field and HundredsDigitAllocationMode to determine value generation methods. The block processes values according to FirstAppearance or Incremental modes while consideringFigureRelationships data for related figure subsets, accessing Parts entity data to retrieve ReservedModuloRemainder values and FirstAppearanceFigure field for initial numbering, and calculating CandidateNumber values while checking Manual Override flags and ManualNumber fields.

[0100] The process flow begins at block S61, which is located at the initial stage and leads to block S63, functioning as the primary data input mechanism for maintaining consistency in handling recurring parts across different images. Block S67 interacts with DocumentSettings to apply numbering scheme rules across related figures and participates in managing fixed numbering schemes within defined figure subsets. Block S69 operates in conjunction with the control block S67 and utilizes part-figure mapping to determine part appearances across figures while maintaining compliance with fixed numbering schemes, with the capability to use ReuseNumbers flag to determine if part numbers should be reused across related figures.

[0101] The process flow initiated at block S61 continues through subsequent blocks S63, S65, S67, and S69, establishing a sequential progression for processing and managing part reference values across multiple images. This sequential arrangement allows for systematic processing of parts data, with each block building upon the data and controls established by previous blocks. The progression through the blocks enables tracking and management of recurring parts while maintaining consistency in reference values across different images.

[0102] The system maintains data consistency by coordinating part reference values between multiple images showing the same parts. When a part appears in different views or contexts, using controlled reference values prevents discrepancies and errors. This coordinated approach enhances the reliability of part documentation across the complete set of images.

[0103] The reuse setting data enables customizable control over how part references are handled between different images. Users can configure how reference values propagate based on specific documentation requirements. This flexibility allows the system to accommodate various documentation standards and workflows.

[0104] Automated propagation of reference values reduces the time and effort needed to manage part references across multiple images. Manual assignment of references for each instance of recurring parts is eliminated. The system automatically coordinatesreferences based on the reuse setting data, streamlining the overall documentation process.

[0105] Figure 15 shows a flow diagram illustrating additional steps for managing part reference values using image relationship information in connection with the process described with reference to Figure 14. The flow diagram depicts a sequence of operations for controlling part reference values across related images, including receiving reuse setting data S71, obtaining part reference values for multiple images S75, S77, and controlling part reference values for recurring parts S81 based on image relationship data S73.

[0106] The process begins at block S71, which receives reuse setting data for controlling part reference values. This reuse setting data may include document-level settings such as DocumentID, numbering scheme configuration, hundreds digit allocation mode, and reserved modulo remainder values, as well as figure-related settings including figure relationship data and FirstAppearanceFigure data. Block S73 then receives image relationship data that defines parent-child relationships between figures, identifying subsets of related images for maintaining consistent part numbering and processing data indicating whether part numbers from parent figures should be reused in child figures.

[0107] The process continues at block S75, which obtains a first set of part reference values for parts appearing in a first image. Block S75 retrieves data from a Parts entity including PartID, PartName, Description fields, as well as FirstAppearanceFigure values, ReservedModuloRemainder values, Manual Override settings, and SortOrder values. Following this, block S77 obtains a second set of part reference values for parts appearing in a second image, processing these values according to FirstAppearance or Incremental modes while respecting figure relationships and accounting for fixed numbering within related figure subsets.

[0108] Decision block S79evaluates whether the first and second images are related based on the received image relationship data. This decision point determines whether to proceed with controlling part reference values and can affect the allocation of hundreds digits when figures are determined to be related. When images are determined to be related, block S81 controls part reference values for recurring parts, making at least lower-order portions of values in the second image correspond to values used inthe first image by generating or otherwise controlling the allocation of values according to the numbering scheme settings. Block S81 maintains part reference value consistency during figure repositioning or modification, and preserves part reference values within defined figure subsets until a part appears outside the related set.

[0109] The process flow integrates handles part-specific settings such as part-figure mapping data, sort order values, and manual override settings for part numbers. Block S73 works in conjunction with the reuse setting data to enable control of part reference values while processing figure identifiers referenced in the relationship data. The system manages candidate numbers from a numbering engine and computed part numbers, with blocks S75 and S77 enable comparison with first set of part reference values for identifying recurring parts, while block S77 considers manual overrides specified for parts and processes values according to ReservedModuloRemainder assignments.

[0110] The process flow operates as part of a larger system for managing relationships between parts across multiple images, with blocks S75 and S77 working together to enable control of corresponding values in related images. These blocks process data from the Parts entity while considering part-figure mapping data that associates parts with specific figures, and provide input data for subsequent comparison and control operations in blocks S79 and S81. Block S79 evaluates whether figures belong to a related subset for numbering purposes and can affect whether FirstAppearance or Incremental numbering mode applies, while also verifying relationships before operations like deletion or repositioning of figures. The system processes part reference values obtained from blocks S75 and S77, determining whether to reuse part numbers from parent figures in child figures based on the ReuseNumbers flag, and functions as a terminal block in the flow diagram with no subsequent operations shown.

[0111] The system may incorporate figure locking functionality and provides control over part-figure mapping data, enabling precise management of part reference values across related images. Block S73 processes timestamp data indicating when relationships between images were created or updated, while block S75 receives input from the image relationship data block and provides output to block S77. The process flow operates within a framework initiated by receiving reuse setting data and utilizingimage relationship data, with block S77 incorporating FirstAppearance mode for establishing consistent part numbering across related figures.

[0112] The decision block S79 maintains a connection with block S73, which supplies the image relationship data needed for relationship analysis. This connection enables verification of image relationships before operations such as deletion or repositioning of figures, allowing the system to maintain data integrity across related image sets.

[0113] The use of image relationship data enables coordinated management of part references across multiple related technical drawings. When drawings show the same component from different views or perspectives, the relationship information allows the system to maintain consistent part reference values. This coordinated approach reduces errors and improves clarity by ensuring that identical parts are labeled consistently throughout the complete set of technical documentation.

[0114] Figure 16 illustrates a flow diagram related to the processes described in Figures 14 and 15, showing steps for generating description text using a neural network language model. The flow diagram depicts a sequence of operations beginning with deriving image relationship input data from image relationships and deriving part input data S83 from a second subset of parts, followed by generating prompt data S85, transmitting the prompt data to a neural network language model S87, and receiving description text data S89 from the language model.

[0115] Block S83 processes various types of part data including PartID, PartName, Description, ReservedModuloRemainder values, and ComputedPartReference, along with part numbering data comprising Manual Override flags and ManualNumber fields. The block accesses part-figure mapping data containing MappingID, PartID, and FigurelD. Block S85 then processes this part input data into a format suitable for the neural network language model, generating candidate part numbers through calculations that multiply FirstAppearanceFigure by 100 and add ReservedModuloRemainder when in FirstAppearance mode, while maintaining fixed candidate numbers for parts within related subsets during Incremental mode operation.

[0116] The method operates through a sequential data processing pipeline where block S83 functions as the initial stage, deriving part input data from a second subset of parts and processing FirstAppearanceFigure data and SortOrder field information. Block S85 receives this part input data and performs recalculation of candidate numbers uponadjustment of part sort order, while generating warnings when figure deletion or repositioning would affect relationship constraints. Block S87 subsequently transmits the generated prompt data to a neural network language model, processing data related to part numbers and handling data associated with Documentsettings parameters, while maintaining relationships between parts and figures through the part-figure mapping entity.

[0117] Block S87 functions as a data transfer mechanism between prompt generation and description text reception, processing information relating to figure relationships and numbering schemes. The block communicates information regarding figure locking status to the neural network model and sends data that defines parameters for language model processing, including prompts that specify requirements for language model output. Block S89 receives description text data output from the neural network large language model, obtaining text descriptions corresponding to a second image based on previously transmitted prompt data, and can store the received description text data in a Parts entity's Description field.

[0118] The system implements a computer-implemented method for processing image data to generate description text. Block S83 handles data from FigureRelationships records and numbering schemes specified in DocumentSettings, while block S85 initiates regeneration of affected fixed part numbers when figure relationships are modified. Block S89 functions as part of this computer-implemented method and acts as the endpoint for collecting the language model's generated textual output.

[0119] The integration of neural network language models enables automated generation of rich technical descriptions based on identified parts and their references. The language model can process the part input data and generate natural, coherent descriptions that accurately capture the relationships between components. This automated approach provides consistent and detailed descriptions while reducing manual effort in documenting technical implementations.

[0120] Figure 17 shows additional steps that can be performed in conjunction with the process described in Figure 14. The flow diagram illustrates a sequence of operations for managing part reference values, beginning with receiving reuse setting data at block S91 and progressing through to receiving user input at block S 101. The operations relate particularly to generating and implementing candidate reference values for recurringparts appearing in multiple images, with steps for previewing and applying the generated values through user interaction.

[0121] Block S91 receives reuse setting data for controlling part reference values, positioned at the start of the process flow and connecting to block S93. The received data encompasses document identification settings including DocumentID for unique identification, NumberingScheme for overall numbering configuration, and HundredsDigitAllocationMode for hundreds digit allocation, along with timestamps for document creation and updates. The block also processes part numbering control settings such as manual override settings, ReservedModuloRemainder values, sort order settings affecting calculations, and part number regeneration rules. Additionally, the block handles figure management settings that define figure relationships for related figure subsets, figure locking settings for numbering integrity, and figure repositioning constraints.

[0122] Block S91 precedes the processing of part reference values for first and second images at blocks S93 and S95, providing control parameters that influence candidate value generation and correspondence between images. Block S93 obtains a first set of part reference values for parts shown in a first image, processing input according to the applicable numbering scheme while analyzing parts using part-figure mapping data. Block S93 retrieves computed part numbers and manual override values, maintaining compliance with defined figure relationships and identifying reserved modulo remainder values for parts in the first image. Block S95 then obtains a second set of part reference values for parts in the second image, utilizing part-figure mapping to identify and collect part information while referencing Part Number History to track changes in reference values between images.

[0123] Block S97 generates candidate part reference values for recurring parts appearing in the second image based on input from the previously obtained reference values. The block can compute candidate numbers through multiple methods, including multiplying FirstAppearanceFigure by 100 and adding ReservedModuloRemainder in FirstAppearance mode, or maintaining fixed values within related subsets in Incremental mode. The generated values are stored in the CandidateNumber field of the Parts entity, with the block determining appropriate values by analyzing figure relationships and part-figure mapping data while enabling recalculation uponadjustment of part sort order. Block S99 then displays a preview of the generated candidate reference value, functioning as an intermediate verification step that shows automatically calculated values incorporating ReservedModuloRemainder and hundreds digit allocation according to the current NumberingScheme.

[0124] Block S99 enables user review and selection of the generated candidate reference value before application, positioned between the generation step and the user input step for adding a callout in the sequential process flow. The preview display reflects active figure relationships and their impact on the numbering, presenting different preview values based on manual override status. Block S101 receives user input for adding a callout containing the generated reference value to the second image, validating the input against figure locking rules and checking if the target figure is locked before allowing callout addition. When new callouts are added, the system updates the FirstAppearanceFigure field, creates timestamp records, and triggers recalculation of candidate numbers while validating positioning relative to parent-child figure relationships.

[0125] The process flow implements a sequential workflow where blocks S93 and S95 function as intermediate processing steps between receiving reuse setting data and generating candidate reference values. Block S93 processes parts according to FirstAppearance or Incremental numbering modes, while block S95 operates within the sequential workflow positioned after receiving reuse setting data and obtaining first reference values. This sequential arrangement maintains consistent part referencing across multiple images while block S97 functions as an intermediate processing step within the workflow for managing reference consistency. The workflow culminates in block S101, which functions as the final step in the sequence beginning with receiving reuse setting data and progressing through generating and previewing reference values, including the capability to process manual override inputs for user-specified part numbers.

[0126] The preview display of candidate part reference values provides an intuitive way to validate numbering choices before final selection. Users can visually confirm that the generated reference values align with their expectations and match the desired numbering scheme. This preview capability reduces potential errors and eliminates rework that would be needed to correct incorrectly applied reference values.Figure 18 shows a flow diagram illustrating steps for managing part reference values in connection with the processes described with reference to Figures 14 and 17. The flow diagram S103-S117 depicts a sequence of operations for handling part reference values, including steps for processing both recurring parts and previously undepicted parts when preparing technical documentation. The operations include receiving reuse setting data, obtaining reference values, generating candidate values, displaying previews, and receiving user input for adding callouts with reference values to images.

[0127] At block SI 03, the process receives reuse setting data for controlling part reference values, which functions as an initial input stage positioned at the top of the diagram. The reuse setting data may include document-level settings such as a document identifier, timestamp information, hundreds digit allocation mode selection, and numbering scheme selection. The process accepts part-specific configuration parameters including part identifiers, names, reserved modulo remainder values, manual override settings, and sort order preferences, along with figure-related settings such as figure identifiers, mapping data between parts and figures, figure relationship data defining parent-child relationships, and figure locking settings for maintaining relationship integrity. The setting data may also include numbering mode configurations, where a FirstAppearance mode derives digits from the lowest-numbered figure, and an Incremental mode maintains fixed part numbers within predetermined subsets, with settings for number reuse between related figures.

[0128] The setting data provided at block SI 03 influences the generation of subsequent part reference values in later processing steps while operating independently from yet procedurally linked to subsequent candidate value generation and display steps. At block SI 05, the process obtains a first set of part reference values for parts depicted in a first image, positioned as a data acquisition step between initial settings receipt and candidate value generation. The process accesses Parts entity data including PartID, PartName, Description, part-figure mapping data, FirstAppearanceFigure data, CandidateNumber values, ComputedPartReference data, ReservedModuloRemainder values, and manual override numbers. Block SI 05 obtains values reflecting figure relationship settings that affect part numbering and provides processed part referencevalues as output for generating candidate values for recurring parts, performing processing specifically focused on the first image in a multi-image processing sequence.

[0129] At block SI 07, the process generates candidate part reference values for parts that recur in a second image, operating as an intermediate processing step between obtaining first set of part reference values and preview display functionality. The generation of candidate numbers can operate in FirstAppearance mode, where candidate numbers are calculated by multiplying FirstAppearanceFigure by 100 and adding ReservedModuloRemainder, or in Incremental mode, where fixed values are maintained within related sets until parts appear outside the related set. The process can update hundreds digits for parts appearing outside related sets and recalculate candidate numbers when part ordering changes, utilizing figure relationship data for determining candidate numbers across related figures and storing generated candidate numbers in a CandidateNumber field.

[0130] At block SI 09, the process displays a preview of the candidate part reference value generated in the previous step, positioned between the candidate value generation step and the user input step for callout addition in the process flow. The preview display enables user review and verification of automatically generated reference values before commitment to drawings, showing different types of preview values including CandidateNumber field values from the numbering engine, numbers derived from FirstAppearanceFigure multiplication by 100 plus ReservedModuloRemainder in FirstAppearance mode, fixed values for parts within related sets in Incremental mode, and manually overridden numbers when Manual Override is active. The process at block Sill receives user input for adding a callout containing the candidate part reference value, enabling association between displayed candidate values and parts in images through part-figure mapping functionality, while processing mapping records containing MappingID, PartID, FigurelD and AddedAt timestamp data.

[0131] The process at block Sill handles candidate number values from the CandidateNumber field of the Parts entity and accesses computed part numbers from ComputedPartReference field when manual override is inactive. When the Manual Override flag is active, the process processes manual number values from the ManualNumber field, and updates the FirstAppearanceFigure field based on the figure where the callout is added. At block SI 13, the process generates candidate partreference values for parts not depicted in existing images, utilizing a predetermined increment from the last-used reference value. The process maintains assigned ReservedModuloRemainder values while generating new candidate values, processes parts appearing in figures outside their related set, and can trigger recalculation of candidate values when part ordering changes.

[0132] The process at block SI 13 operates sequentially after the generation of candidate values for recurring parts, providing output to the preview display at block SI 15 and coordinating with user input handling at block SI 17. Block SI 15 displays a preview of the generated candidate part reference value for an undepicted part, positioned between the generation step and user input for adding a callout in the workflow sequence. The preview display updates in response to changes in part sort order, modifications to figure relationships affecting numbering, manual override adjustments, and recalculations triggered by reordering parts, presenting candidate values computed through either incremental mode or first appearance mode to enable visual confirmation before implementation.

[0133] At block SI 17, the process receives user input for adding a callout containing a further candidate part reference value to a second image, functioning as the final step in the sequence following the generation and preview of the further candidate value. The process interfaces with the Parts entity to access candidate numbers based on FirstAppearance or Incremental mode, recording the timestamp of callout addition through the AddedAt field in part-figure mapping. Block SI 17 forms part of an iterative process for handling undepicted parts with candidate values. This step complements the earlier user input step at block SI 11, which operates on initial candidate values, while triggering updates to part-figure mapping when a new callout is added.

[0134] The process at block SI 13 participates in an iterative process of generating and previewing candidate values, with each iteration enabling refinement of the reference numbering scheme. The preview display functionality at block SI 15 supports this iterative approach by presenting values derived from both the incremental mode calculations and the first appearance mode processing. Block SI 15 updates the preview display when changes occur in the reference value components, including modifications to hundreds digits, providing continuous visual feedback throughout the iterative numbering process.The process at block SI 13 incorporates functionality to update reference value components, including hundreds digits, when generating candidate values for undepicted parts. This updating capability enables the reference numbering system to maintain consistent numerical patterns while accommodating new parts that appear in different sections or related groups of figures. The updating of hundreds digits operates in coordination with the predetermined increment mechanism to ensure that newly generated reference values align with the established numbering scheme while preserving the logical structure of part relationships across multiple images.

[0135] The automated generation of sequential reference values for newly depicted parts helps maintain consistent and organized numbering throughout technical documentation. When a previously undepicted part appears in a subsequent image, the system automatically suggests the next available reference value based on predetermined increments from previously used values. This automated approach reduces manual effort in reference value assignment while preventing errors such as duplicate numbers or gaps in the sequence.

[0136] Figure 19 shows a flow diagram illustrating additional steps for controlling part reference value reuse across multiple images in connection with the process described with reference to Figure 14. The flow diagram depicts a sequence of operations for managing part reference values, particularly where parts recur across different images. The process includes receiving reuse setting data SI 19, obtaining part reference values S 121 , S125, and controlling value reuse based on various decision points S123, S127, S129 to determine appropriate part reference assignments across multiple images.

[0137] At block SI 19, the process receives reuse setting data including element-level reuse control data that influences decisions about part reference value reuse across images. The reuse setting data may include document identification data, numbering scheme parameters, and hundreds digit allocation mode settings, along with figure-related data such as reuse number flags, parent-child relationships between figures, figure locking status information, and part-figure mapping data. The received data may also include part-specific information encompassing part identifiers, part names, part number history data, and manual override settings for part numbers, as well as timestamps for creation and updates. Block S 121 then obtains a first set of part reference values for parts appearing in a first image, accessing multiple data types includingPartID and PartName attributes, FirstAppearanceFigure values, ReservedModuloRemainder values, CandidateNumber values, ComputedPartReference values, and Manual Override values.

[0138] The process flow continues at decision block S123, which obtains and evaluates element-level reuse control data for a second image. This decision block accesses partfigure mapping and FigureRelationships data that defines relationships between parent and child figures, determining whether numbering from parent figures can be reused in child figures based on a ReuseNumbers flag. Block S125 then initiates the generation of part reference values for parts appearing in the second image, accessing parts entity data including PartID, PartName, Description, ComputedPartReference, and CandidateNumber, while also evaluating figure relationships through FigureRelationships records to determine appropriate numbering schemes.

[0139] The process proceeds to decision block S127 which evaluates whether a part appearing in the second image is a recurring part that also appeared in the first image, functioning as a conditional branch point for determining how to handle part reference values between images. In block S128, if the part is not recurring, the flow returns to block S125. Otherwise, following this, decision block S129 determines whether to reuse at least a modulo remainder portion of a part reference value from a preceding image, with this determination based on the element-level reuse control data received earlier in the process. If at least the modulo remainder portion is not to be re-used, the flow returns, via block S128, to block SI 25. Otherwise the process continues to block S131, which controls and adjusts part reference values in the second image to correspond with values from the first image, enabling fixed numbering schemes within predetermined subsets of related figures while updating and recalculating part reference values when figure relationships change between parent and child figures. The next part reference value generation is then initiated at block S125, or if no parts to be allocated reference values are left, the process concludes.

[0140] The process controls part reference values across related images. Block S121 follows the reuse setting data reception of block SI 19 and leads to block S123, operating within a system that can selectively reuse modulo remainder portions of part reference values. The system utilizes part-figure mapping to determine parts associated with the first image and accesses Documentsettings to determine the numberingscheme, supporting both FirstAppearance and Incremental numbering modes. Decision block S127 can affect the calculation of CandidateNumber fields and determine whether hundreds digits remain fixed or increment for parts.

[0141] The process implements several mechanisms for managing figure relationships and part reference values across the workflow. Block S123 precedes determinations about recurring parts and modulo remainder portions, while evaluating figure relationships to determine numbering subset membership. Block S125 functions as an intermediate step between initial part reference value collection and final value control operations, analyzing FirstAppearanceFigure fields and verifying ReservedModuloRemainder values for parts while checking for manual override numbers and hundreds digit allocation modes. The system processes reuse setting data related to modulo remainder portions of part reference values, with block S131 receiving input flow from decision block S129 and utilizing FirstAppearanceFigure field and CandidateNumber field data to determine appropriate numbering while referencing part-figure mapping to identify parts appearing across multiple figures.

[0142] The process flow incorporates flexible mechanisms for managing part reference values and their reuse across images. Decision block S 123 evaluates figure relationships to determine whether fixed part numbering should be maintained across related figures, while also relating to control data that specifies whether modulo remainder portions of reference values can be reused between images. Block S125 maintains timestamp information indicating part addition or updates in the second image and enables subsequent analysis of modulo remainder portions of part reference values. When evaluating parts in the second image, block S127 operates in conjunction with the FigureRelationships entity to establish numbering rules and determine number reuse across related figures, while decision block S129 implements element-level reuse control records and determines whether the two-digit remainder modulo 100 remains constant for related parts.

[0143] The process flow concludes with coordinated operations across multiple blocks to ensure consistent part reference value management. Block S127 can influence FirstAppearanceFigure values and trigger recalculation of part numbers when parts appear in multiple figures. Decision block S129 leads to block S131, which processes first set reference values from block S 121 while operating within a system that supportsFirstAppearance and Incremental numbering modes. Block S131 implements controls for part reference value correspondence between images.

[0144] The granular control over part reference value reuse enables flexible management of part numbering across different images. By selectively reusing modulo remainder portions for recurring parts, logical groupings can be maintained while allowing different numbering schemes to be applied as needed. This approach supports consistent part identification while accommodating varying organizational requirements between images.

[0145] Figure 20 illustrates additional processing steps related to the handling of part reference values shown in Figure 14. The flow diagram depicts a process for managing part reference values across multiple images based on received reuse setting data, including numbering mode settings. The process involves obtaining part reference values from first and second images, determining whether parts recur across images, and controlling the reference data values to maintain consistency between images.

[0146] Block S133 receives reuse setting data comprising numbering mode settings that determine how part numbers are allocated across multiple images. The settings include HundredsDigitAllocationMode parameters that specify either FirstAppearance or Incremental modes, along with document identifier settings and timestamp data for creation and updates. These reuse settings establish numbering parameters that affect how recurring parts are handled across multiple images and provide configuration information that influences part reference control in subsequent steps. The received reuse settings can be applied to individual images, subsets of images, or entire image sets, allowing flexible control over the numbering scheme implementation.

[0147] Block S135 obtains a first set of part reference values from parts appearing in a first image, receiving input from the setting data established in block S133 and providing output to block SI 37. The block processes part data including PartID, PartName, Description, and ReservedModuloRemainder values while determining figures in which parts appear and computing FirstAppearanceFigure values. Part data processing accommodates both FirstAppearance and Incremental numbering modes, handling parts with manual overrides and fixed numbers within related figure subsets while maintaining part-figure mappings and timestamp data for parts appearing across multiple related figures.Block SI 37 then initiates the generation of a second set of part reference values for parts present in a second image, functioning sequentially after block S135 and preceding decision block S139. The block accesses part information including part IDs, names, descriptions and computed numbers for elements in the second image, while retrieving part-figure mapping data that links parts to their appearance in the second figure. Block SI 37 retrieves figure relationship data indicating whether numbering should be reused from a parent figure and accesses document settings that determine part number allocation across figures, operating within the system governed by numbering mode settings established in block SI 33.

[0148] Decision block S139 dermines whether part(s) to be depicted in the second image correspond to parts from the first image. In block S140, if the part is not recurring, the flow returns to block S125. Otherwise, following block S139, flow moves to block S141 in which FirstAppearanceFigure values and CandidateNumber are calculated based on the part's presence in related figures, utilizing part-figure mapping data to track these appearances. Block S141 subsequently controls part reference values, processing reference data according to the established numbering mode settings while applying appropriate schemes for individual images, subsets of images, or entire image sets. The control operations update reference values based on figure relationships, FirstAppearanceFigure calculations, ReservedModuloRemainder values, manual overrides, and Part-Figure mapping validations, ensuring consistent part referencing across multiple images.

[0149] Block S141 maintains data consistency by recalculating reference values when part ordering changes within or across figures. This ensuring that part references maintain their integrity even as documentation evolves. This recalculation capability, combined with the Part-Figure mapping validations, allows the system to adapt to modifications in technical documentation while preserving consistent part identification across multiple images. The next part reference value generation is then initiated at block SI 37, or if no parts to be allocated reference values are left, the process concludes.

[0150] The configurable numbering mode settings enable flexible control over part reference values at multiple levels of granularity. Settings can be applied independently to individual images, subsets of related images, or across entire image sets. This multi-level control allows documentation to be structured according to varying requirements, such as using different numbering schemes for different views or sections while maintaining consistency where needed.

[0151] Figure 21 shows a flow diagram illustrating steps for processing part reference values based on numbering mode settings, and relates to the processes described with reference to Figures 14 and 20. The flow diagram S143-S159 depicts a sequence of operations for handling part references across multiple images, including steps for determining whether parts are recurring and controlling how reference values are generated based on various numbering mode settings.

[0152] The process begins at block S143, which receives reuse setting data including numbering mode settings and document-related configuration data. The received reuse settings may specifications for high-order number portions of part reference values, numbering modes such as FirstAppearance mode and Incremental mode, and DocumentID and NumberingScheme parameters. Block S145 follows, obtaining a first set of part reference values for parts appearing in a first image by accessing Parts entity data including PartID, PartName, Description, ComputedPartReference, and ManualNumber fields. The process then advances to block S147, which then initiates the generation of part reference values for parts appearing in the second image, utilizing part-figure mapping to identify and locate parts across different figures.

[0153] A decision point S149 evaluates whether a part appears in multiple images, branching to either a process for generating independent part reference values or controlling recurring part references. For non-recurring parts, block S151 generates independent reference values by calculating candidate numbers based on either FirstAppearanceFigure multiplication by 100 plus ReservedModuloRemainder in FirstAppearance mode, or using subset-based fixed values in Incremental mode. When parts are determined to be recurring, block SI 53 controls reference values using reuse setting data and first image values as inputs, processing reference values according to system parameters while considering manual override settings and validation against reserved modulo remainder constraints.

[0154] Decision block SI 55 determines whether a high-order numbering mode is active, with two output paths leading to different reference value generation methods. When high-order numbering is active, block SI 59 generates part reference values byincorporating the image index into the high-order portion, calculating candidate numbers by combining this high-order portion with reserved modulo remainder values in either FirstAppearance or Incremental modes. Alternatively, block SI 57 generates reference values without using image index for the high-order portion, maintaining fixed part numbers within related figure subsets and incrementing hundreds digits when parts appear in unrelated figures outside the fixed subset. The next part reference value generation is then initiated at block S147, or if no parts to be allocated reference values are left, the process concludes.

[0155] The system provides configuration data that influences subsequent part reference value generation steps, particularly at decision point SI 55 regarding high-order numbering mode, determining whether standard or high-order numbering modes will be used in subsequent processing steps. Block S147 receives inputs from both independent part reference value generation processes and controlled part reference value processes, accessing stored part information including ComputedPartReference and ManualNumber fields through the Parts entity. The decision point at block S149 enables determination of whether parts require coordinated numbering across multiple figures, supporting fixed numbering within predetermined subsets of related figures and affecting candidate number calculations when parts appear across related or unrelated figures.

[0156] The processing workflow incorporates sophisticated figure relationship management through FigureRelationships entity data, enabling the system to determine related figures and establish numbering subsets while maintaining FirstAppearanceFigure values for parts appearing across multiple figures. Block S151 and block SI 53 outputs to block SI 55 for high-order numbering mode evaluation and connects to subsequent operations that generate part reference values. The system tracks part number history with creation and update timestamps, supports part identification through PartID assignment, and enables part-figure mapping modifications, with block SI 57 recalculating CandidateNumber values when part reordering occurs and regenerating affected fixed part numbers when figure relationships change.

[0157] The system supports different numbering modes through block S147, which operates within a system capable of implementing varied numbering modes including high-order numbering based on image indices. Block S149 supports FirstAppearancemode and Incremental mode while forming part of a workflow for managing and generating part reference values, with the decision block affecting how hundreds digits remain fixed or increment for parts in unrelated figures. Block S151 operates in conjunction with numbering mode settings received in block S143, computing FirstAppearanceFigure values while considering figure relationships, and functioning independently of image index-based reference value generation in contrast with blocks S157 and S159.

[0158] Block SI 59 implements multiple operational modes that provide flexible control over part reference value generation. In FirstAppearance mode, the digits derive from the lowest-numbered figure, while in Incremental mode, numbers remain fixed within predetermined subsets, with the decision block managing the allocation of hundreds digits in part numbers based on the selected mode. The calculation method for the CandidateNumber field varies according to these modes, using either FirstAppearanceFigure multiplication or subset-based fixed values, which enables block SI 57 to function as an alternative to block SI 59 for generating reference values.

[0159] The high-order numbering mode enables systematic organization of related parts through controlled management of high-order number portions. Parts can be logically grouped by assigning specific high-order values while maintaining unique individual references through lower-order portions. This structured approach to part reference numbering supports efficient categorization and identification of components across multiple related assemblies or subsystems.

[0160] Processing High-Order Part Reference Values Across Multiple Images Figure 22 illustrates additional processing steps that can be performed in relation to the process shown in Figure 14. The flow diagram depicts a method for controlling part reference values across multiple images, particularly focusing on handling high-order numbering modes and recurring parts. The process flow begins with receiving reuse setting data S161 and proceeds through various decision points and processing steps to determine appropriate part reference values and apply any necessary adjustments SI 75.

[0161] Block S161 receives reuse setting data that controls part reference values, processing DocumentID settings for unique document identification andNumb ering Scheme settings that specify overall numbering arrangements. The block processes reuse settings for HundredsDigitAllocationMode and manual override options for part numbers, while handling reuse settings that define figure relationships and determine how parts are numbered across related figures. Block SI 63 subsequently obtains part reference values for components appearing in a first image, accessing stored part data and part-figure mapping to retrieve reference values associated with parts in the first image, including reserved modulo remainder values and FirstAppearanceFigure values for parts shown in the first image.

[0162] Block SI 65 then initiates the generation of part reference values for parts appearing in the second image. Block 167 evaluates whether a high-order numbering mode is set, directing the process flow along two possible paths: an affirmative path leading to block SI 69 for high-order number generation, or a negative path leading directly to block S175 for obtaining additional reference values. The decision determines whether the numbering scheme uses FirstAppearance mode, where the hundreds digit is derived from the lowest-numbered figure, or Incremental mode, where values are updated based on figure relationships. Block SI 69 generates the high-order number portion according to the selected mode, with FirstAppearance mode deriving values by multiplying the FirstAppearanceFigure by 100, while Incremental mode maintains fixed values within predetermined subsets of related figures and updates when parts appear outside the related set.

[0163] Decision block S171 then evaluates whether a part appears in multiple images, with two possible output paths: a positive path leading to part reference value control in block SI 73, and a negative path returning to block SI 65 for processing additional parts.

[0164] Block SI 73 processes part reference values for recurring parts by utilizing reuse setting data and existing values from the first image, functioning as an intermediate processing step before potential incremental adjustments. The block processes reference values based on defined subsets of images according to element-level reuse control data records while maintaining consistency in part referencing across multiple images.

[0165] Block S175 applies incremental adjustments to part reference values based on specified mode settings, modifying the hundreds digit of part numbers when operatingin Incremental mode while maintaining fixed reference values for recurring parts within defined image subsets and updating part numbers when parts appear in figures outside a defined subset.

[0166] The process flow incorporates several mechanisms for managing part reference values across image sets. Block S161 processes sort order settings and ReservedModuloRemainder values affecting part number calculations while accepting configuration data for controlling incremental high-order numbering modes and value reuse across image subsets. Block SI 63 establishes baseline reference values that may be modified based on subsequent numbering mode settings, providing input values for potential adjustment based on image indices or incremental modes. The CandidateNumber values are calculated for parts appearing across multiple images, with the system maintaining consistency through mechanisms such as fixed reference values within defined image subsets, incremented values, and potential reuse of numbering from parent figures based on the ReuseNumbers flag.

[0167] The process flow implements interconnected data paths between blocks to facilitate efficient processing of part reference values. Block S161 provides input data that influences processing in blocks SI 63 and SI 69. Block SI 67 processes downstream of decision block SI 65 and provides input to block SI 69 for generating part reference values, while operating within a system that handles recurring parts across multiple images. Block SI 73 interfaces with document settings and figure data while considering related figure sets, utilizing timestamp data and interacting with part-figure mapping systems when processing control values. Block S 175 receives input directly from block SI 73 and operates after determining part recurrence, functioning in conjunction with high-order numbering mode settings established earlier in the process.

[0168] The process incorporates flexible control mechanisms for managing part reference values based on image indices and relationships. Block SI 67 generates numbers based on image index associations while processing numbering according to mode settings associated with entire sets of images, with the capability to recalculate numbers when part sort orders are adjusted or when figure relationships affecting the numbering subset are altered. Block SI 69 operates within a system utilizing image indices to track parts across multiple images, connecting to a subsequent decision point that evaluates whether a part is recurring, with potential loop-back functionality tosupport both incremental and image-index based numbering schemes. Decision block S171 functions within a broader system for managing part reference numbering across multiple images and related figure sets, particularly applicable to incremental high-order numbering mode where recurring parts receive dynamically controlled reference values.

[0169] The process flow implements adaptive handling of part reference values based on various control mechanisms. When a part is determined to be recurring, the CandidateNumber field calculation adapts according to the selected mode settings, with the system maintaining consistency through the part-figure mapping. Block SI 73 operates within a high-order numbering control process for managing part reference values, interfacing with the broader system to process values based on defined subsets of images while accommodating controlled variations. Block S167 can process manual overrides to bypass automatic number generation, providing flexibility in cases where standard numbering schemes require adjustment.

[0170] The process flow implements conditional branching based on the high-order numbering mode status, with block SI 63 outputting values that lead to different processing paths depending on the mode setting in decision block SI 65. When high-order numbering mode is active, the system influences how parts are handled across related figures, affecting whether part reference values remain fixed within related subsets or are updated when appearing in unrelated figures. Block SI 67 forms part of this conditional branch in the flowchart, with an alternative path bypassing the block when the mode is not set, while block S175 functions in conjunction with these high-order numbering mode settings to maintain consistency in the numbering scheme.

[0171] The generation of high-order number portions based on image indices creates a logical and intuitive structure for part references throughout a document. The correlation between part reference values and their corresponding image positions helps users quickly identify and locate specific components.

[0172] The incremental high-order numbering mode enables flexible organization of part references that operates independently from the sequential order of images. This approach allows part references to follow alternative organizational schemes based on logical groupings or functional relationships rather than being constrained by image placement.The use of defined image subsets for maintaining fixed reference values creates clear organizational boundaries within technical documentation. Parts that appear multiple times within a related group of images retain consistent reference values, making it easier to track components across logically connected views. When parts appear outside their defined subset, the incremented high-order numbering provides distinct identification.

[0173] The ability to define multiple element-level reuse control records, above the part or group-of-parts level, enables relatively precise management of reference behavior across distinct images or image groups within a document. Each control record can specify a different subset of images with customized reference value handling. This granular configuration supports varied reference value requirements across different sections while maintaining overall document consistency.

[0174] The system supports flexible configuration of image subsets through elementlevel reuse control records associated with the document's images. Different subsets can be defined to reflect logical groupings, functional relationships, or other organizational needs. The independent configuration of multiple subsets allows reference values to be managed according to the specific context and requirements of each image group.

[0175] The high-order numbering mode settings operate at a document-wide level while allowing granular control through multiple value reuse records. This combination enables consistent overall numbering schemes while supporting localized variations in reference value handling. The parallel operation of document-level settings and subsetspecific controls provides both consistency and flexibility in reference value management.

[0176] Figure 23 shows a flow diagram illustrating steps for maintaining part reference data during image modification operations and relates to features described in connection with Figure 14. The flow diagram S193 depicts a process for automatically updating part reference values in response to changes in a set of images, including handling requests for image deletion and image reordering operations while preserving referential integrity between parent and child images.

[0177] The process begins at decision block SI 77, which checks whether an image deletion has been requested. When a deletion is requested, the flow proceeds to decisionblock SI 79 to determine whether the selected image is designated as a parent figure by checking ParentFigurelD fields in FigureRelationships records. If the image is identified as a parent, block SI 81 inhibits the deletion and can display a warning message to preserve figure relationships; if not, block SI 83 allows the deletion to proceed, provided there are no dependency conflicts. Both paths subsequently feed into the part reference maintenance function at block SI 93.

[0178] When an image deletion is not requested, the flow proceeds along the 'No' path from block SI 77 to decision block SI 85, which evaluates whether an image reorder operation has been requested. If reordering is requested, the process advances to decision block SI 87, which checks whether the proposed reordering would position a child image before its parent image in the sequence by comparing ChildFigurelD information against ParentFigurelD positions. Based on this evaluation, the process directs to either block SI 89 to inhibit reordering or block S191 to allow reordering, with both paths subsequently feeding into the part reference maintenance function at block SI 93.

[0179] The part reference maintenance function at block SI 93 processes inputs from both deletion and reordering paths to update part reference values automatically. When modifications occur, the system recalculates CandidateNumber values for affected parts while maintaining ReservedModuloRemainder constraints and updates FirstAppearanceFigure fields based on changes in part-figure mapping relationships. The function preserves fixed numbering schemes across related figures and can trigger regeneration of affected part numbers when figure relationships are modified, with modifications being recorded in the Part Number History entity along with associated timestamps. This automated processing continues unless a manual override is active, ensuring consistent part reference data maintenance across the image set while protecting parent-child relationships and sequential ordering constraints.

[0180] The workflow implements multiple protection mechanisms to maintain data integrity across figure relationships. The figure locking functionality prevents deletion of parent figures that have existing relationships with child figures, while also protecting fixed numbering schemes across related images. When reordering is permitted through block S 191 , the system adjusts the SortOrder field in the Parts entity and triggers recalculation of affected part numbers while maintaining establishedrelationship constraints. The process includes validation steps that protect parent-child relationships during both deletion and reordering operations, with warning messages displayed to users when attempting operations that would compromise these relationships.

[0181] The workflow integrates with a broader part reference maintenance function at block SI 93 that coordinates both deletion and reordering operations through interconnected decision points. When processing deletion requests, the system checks relationships between blocks SI 77 and SI 79 to evaluate parent-child dependencies, while reordering operations are managed through blocks SI 85 and SI 87 to maintain sequential ordering. Block SI 89 and block S191 operate as complementary paths within the workflow, with block SI 89 preventing repositioning of child figures to positions earlier than their parent figures, and block S 191 enabling modification of part sequences without disrupting underlying numbering rules. This integrated approach ensures that the workflow maintains consistency across the Parts entity while preserving established relationships between figures.

[0182] The workflow incorporates additional safeguards when processing image operations through block SI 93. When images are reordered, the system updates FirstAppearanceFigure values to reflect the new sequence positions while maintaining strict adherence to parent-child precedence rules. The decision points at blocks SI 79 and SI 87 operate in conjunction with protection mechanisms that prevent both the deletion of parent figures having dependent relationships and the repositioning of figures in ways that would violate established ordering constraints. These protection mechanisms extend across the entire workflow, with block S 189 specifically preventing operations that could compromise the integrity of fixed numbering schemes.

[0183] The workflow implements additional features to maintain consistency when figures are modified. The process at block SI 93 operates within a system that begins with an image deletion request check at block SI 77, coordinating with subsequent blocks to enforce ordering and relationship constraints between parent and child images. When reordering operations are processed through the parallel branch starting at block SI 85, the system evaluates potential impacts on fixed numbering schemes across related figures, ensuring that modifications preserve established relationships while enabling appropriate adjustments to figure sequences.The automatic maintenance of part reference values eliminates the need for manual updates when changes are made to the image set. This automated propagation helps prevent inconsistencies and errors that could arise from manual reference management.

[0184] The system protects referential integrity by preventing deletion of parent images that are used to define part reference values in child images. When a user attempts to delete a parent image that contains referenced elements, the system blocks the deletion operation. This safeguard eliminates the risk of creating orphaned references or invalid part data values that could compromise data consistency across related images.

[0185] The system maintains logical ordering relationships between parent and child images during reordering operations. When a user attempts to reorder images, the system prevents child images from being positioned before their associated parent images in the sequence.

[0186] Reordering and Recalculating Part reference values

[0187] Figure 24 relates to the features shown in Figure 14 and illustrates additional optional steps in a flow diagram for controlling part reference values. The flow diagram shows a process beginning with receiving reuse setting data at block SI 95, followed by obtaining part reference values from first and second images at blocks SI 97 and SI 99, respectively. A decision point S201 determines whether to reorder parts, leading to either reordering and recalculation of reference data values through blocks S203 and S205, or maintaining initially-generated candidate values at block S207.

[0188] Block SI 95 receives reuse setting data that influences subsequent part reference value calculations throughout the process flow. The reuse setting data may include document identifiers, numbering scheme parameters, hundreds digit allocation modes, timestamp data, appearance mode settings, manual override settings, reserved modulo remainder values, figure relationship settings, sort order preferences, and figure locking settings. Block SI 97 obtains a first set of part reference values for parts shown in a first image, retrieving data values including part identifiers, names, reserved modulo remainder numbers, manual override flags, first appearance figure information, candidate numbers, and computed part numbers from the parts entity, part-figure mapping records, and figure relationship records. Block SI 99 obtains a second set ofpart reference values for parts shown in a second image, accessing stored part data including unique identifiers, names, descriptions, and numbering information while retrieving part-figure mapping data to determine part appearances across figures.

[0189] The reordering capability with automatic recalculation of part references maintains consistency throughout document modifications. When parts are reordered, the system automatically updates reference values based on the current reuse setting data. This automated reference adaptation eliminates manual renumbering effort while preserving proper sequential numbering across the document structure.

[0190] Figure 25 illustrates the third web app window 81, as illustrated in Figure 10, in which the image editor 45 is embedded, when a second image is being drawn and / or annotated with callouts. Part references in this example are controlled in accordance with the methods described in relation to Figures 14 to 24. As mentioned, the initial candidate part references displayed as a preview can be one, or both, of two kinds: fixed part references, and dynamic part references. In the example shown in Figure 24, various parts have been allocated to the image and given the part reference values 202, 204, ... 220 that are shown in first sub-region 87. Remaining parts have not yet been allocated to the image and have each been given either the dynamic candidate part reference value 222, or the fixed part reference value 214, as shown in the second subregion 88. The fixed part reference value in this example is generated using the image index (the image in this case is Figure 2) for the hundreds position due to the numbering mode selected in this example, and the modulo remainder part is reused from the corresponding part reference value (114) used for the same part in the first image shown in Figure 10. If any one of the remaining parts is allocated to the image by adding a callout, the relevant candidate part reference value, i.e. 222 or 214, is added to the callout in the image and associated with the part as shown in the image, and the relevant part is moved to the first sub-region 87. Furthermore, if the relevant candidate part reference value is the dynamic candidate part reference value 222, the remaining part dynamic candidate part reference value is updated to 224 (the predetermined increment being 2 in this example.) If the relevant candidate part reference value is the fixed candidate part reference value 214, the remaining part dynamic candidate part reference value is kept at 222.In Figure 25, the figure relationship setting has used. As shown in the GUI element 92 displayed in the second sub-region 89, the user having specified one or more related images, being parent images, for the current image. In the example shown in Figure 10, the image being generated is “Figure 2” and “Figure 1” being available. In association with the figure relationship setting, the reuse flag can be set using reuse flag GUI element 93.

[0191] Figure 26 shows a fourth web app window 101 showing text generated for the image of the specific example (“Figure 2”) as illustrated in Figure 25, once completed. The generated text may then be edited by the user to correct any errors introduced by the neural network LLM system 5.

[0192] Modifications and Further Examples

[0193] In the document generation system described above, the determination of the subset of parts present in an image is based on user input. In an alternative example, the determination could be performed at least in part automatically by using image analysis software to automatically identify parts, for example using an object segmentation functionality and object recognition functionality.

[0194] In some examples, as shown in Figure 13, image data for an image is processed, at S52, using image analysis software and parts in the set of parts that are likely to be present in the image are identified, at S53, based on the image analysis. In an example, the image analysis software employs GPT-4V(ision) to enable a neural network LLM to analyse image data and generate image text data providing a description of an image. That image text data can then be compared to the set of parts to identify matches, with the matched parts indicating parts in the set of parts that are likely to be in the image. The match can be determined based on a set of rules encompassing both identical word matching and synonym matching. The subset of parts is then populated, at S55, with the identified parts and the subset of parts is displayed, at S57, to the user. The user can then amend, at S59, the subset of parts to remove parts not in the image and to add parts that have not been identified by the image analysis software.

[0195] While the automatic document generation system of Figure l is a web-based system that is accessed by a user using a web browser, it will be appreciated that other configurations are possible. For example, the front end could be provided by anapplication running on a user device, with that application communicating with a database and worker process provided in the cloud. It is also possible for the front end, database, and worker process to all be implemented in a user device, however it is envisaged that the database 9 and worker process 11 will be based in the cloud and handle interactions with many different users.

[0196] In the above-described examples, the NLP processing system 3 is separate from the platform 1. Alternatively, the NLP processing system 3 can be implemented on the platform 1, in which case the output from the NLP pipeline 17 can be stored directly in the database 9 making the NLP model storage 19 and the serialiser 21 redundant.

[0197] In the above-described examples, the prompt data, which is sent to the neural network LLM before receiving description text data for the image in return, comprises textual data in the form of the description data and the part data. In alternative examples, once editing of the image has been completed and all parts in the image have been added, the image including all the reference numerals shown as callouts may also be sent to the neural network LLM as part of the prompt data. The neural network LLM may have an image analysis component, such as GPT-4V(ision), to enable the neural network LLM to analyse the image, as instructed in the prompt data, along with the description data and the part data, to enhance the resulting textual description of the image. Since the image includes the part reference numerals and the parts list is given with the corresponding reference numerals, the neural network LLM is able to assign the correctly assigned part names, along with the correctly assigned reference numerals, when describing features from the image which results from a textual description of the image generated by the image analysis component.

[0198] In the above-described examples, the part data is generated by natural language processing of unstructured text data to generate corresponding structured text data and processing the structured text data to generate a set of parts. A user may edit the structured text data or the set of parts that is generated from the structured text data. In alternative examples, the user may manually enter, or select from one or more suggested part names, part names during an image editing process, e.g. when adding callouts to the images when using the image editing software, thus obviating the need for, or alternatively supplementing, the processing of the unstructured text data as part of the process to generate the set of parts. The subset of the set of parts that are associatedwith each image is then stored against each image and used to formulate the prompt data for the generation of the description of that particular image. The process may be repeated for each image, and the prompt data generated on the basis of the resulting part data, may be sent after each respective image is drawn and / or edited with callouts or at the end of the image editing process. The resulting part data for each image, may be included in prompt data for generating a description of the image along with textual description data, for example patent claim text data identified, by the user or by mapping from the subset of parts to the relevant claims, to be relevant to the image and / or descriptive text about what is shown in the image.

[0199] While the described NLP processing system 3 utilises routines from the spaCy library, it will be appreciated that alternative routines performing substantially the same function could be used. It will also be appreciated that the heuristic rules applied by the NLP pipeline 17 may be modified based on knowledge of the nature of an image. For example, when the image is for a patent specification and the originally entered text is a set of patent claims, the formatting that is specific to a set of patent claims, e.g., the presence of claim numbers and claim dependencies, can be taken into account in the heuristic rules.

[0200] Although the neural network LLM system 5 may be implemented in an external system and accessed via an API, in alternative embodiments the neural network LLM system may be a neural network LLM internally hosted on the platform. While the described system uses the ChatGPT™ API to access a neural network LLM based on GPT-4, alternative neural network LLM models could be used, for example Google Gemini™ and LLaMa™ by Meta.

[0201] While a database 9 is used in the document generation system of Figure 1 to facilitate parallel processing of interactions with many users, particularly given the time delays associated with the NLP pipeline and the neural network LLM system, in alternative embodiments it is possible to use conventional memory management techniques instead.

[0202] Illustrations of web app windows for an example implementation have been provided to assist explanation. These web app windows include graphical user interface (GUI) elements, such as buttons and text boxes, that afford the opportunity for a user to interact with the web app window (such GUI elements are sometimes referred to asaffordances). It will be appreciated that the design of the web app windows could be altered and the affordances replaced with affordances with similar functionality without substantially altering the functionality of the document generation system.

[0203] The specific examples provided relate to the generation of a patent specification, with the originally received unstructured text data corresponding to a set of patent claims and the image being a patent figure, with the neural network LLM being used in the generation of a description for the patent figure. The platform may allow a user to input a sequence of patent figures in relation to the same set of claims, as is commonly present patent specifications. As at least at present, neural network LLMs provide best results when the prompt data is both specific and concise, in an example the document generation system iteratively generates image by image text for the sequence of patent figures, with the text for an image being generated as described above, and then concatenates the text for the images to generate a detailed description of the images for the patent specification. In addition, prompt data can also be provided to the neural network LLM to generate appropriate background and summary sections so that an entire patent specification can be prepared following generation of a set of patent claims.

[0204] Alternative applications of the document generation system can be in the preparation of academic papers or the preparation of technical reports. For all applications, the originally received text data describes entities that can be associated with parts shown in an image.

[0205] It will be appreciated that while the document generation system employs generative Al to generate text content, and that many measures are employed to engineer prompt data that improves the accuracy of the generated text content, the ability for user review and editing is built into the document generation system to allow the opportunity for a user to correct errors that can be introduced when using natural language processing and generative Al.

[0206] According to an independent aspect of the invention, which may be combined with the features of any one or more of the accompanying claims and / or aspects and / or features of the examples described above, there is provided a computer-implemented method of obtaining part reference values for parts in a set of images, comprising:allocating a first set of part reference values for a first subset of parts in a first image in the set of images; and

[0207] identifying at least one part that is in the first image and recurs in a second image in the set of images; and

[0208] for said at least one part, allocating a part reference value in a second set of part reference values for a second subset of parts in the second image, wherein at least a portion of said part reference value is automatically reused from a corresponding part reference value in the first set of part reference values.

[0209] The above examples are to be understood as illustrative examples of the invention. It is to be understood that any feature described in relation to any one example may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the examples, or any combination of any other of the examples. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims

CLAIMS1. A computer-implemented method of obtaining part reference values for parts depicted in a set of images, the method comprising:receiving reuse setting data to at least partially control at least some of a part reference value for a recurring part that is depicted in more than one image;obtaining a first set of part reference values for a first subset of parts depicted in a first image in the set of images; andobtaining a second set of part reference values for a second subset of parts depicted in a second image in the set of images, wherein at least a portion of a part reference value for a recurring part when depicted in the second image is at least partially controlled, in dependence on the reuse setting data, to accord with at least a corresponding portion of a part reference value for the recurring part when depicted in the first image.

2. A computer-implemented method according to claim 1, wherein the method comprises:receiving image relationship data defining relationships between images, the image relationship data defining a subset of related images in the set of images, wherein the at least one part reference value is controlled in dependence on the image relationship data indicating a relationship between the second image and the first image.

3. A computer-implemented method according to claim 1 or 2, further comprising formulating prompt data for a neural network large language model to generate description text corresponding to the second image, the formulating of the prompt data comprising:deriving part input data corresponding to the second subset of parts depicted in the second image; andgenerating prompt data comprising the part input data;transmitting the prompt data to the neural network large language model; andreceiving description text data for the image from the neural network large language model.

4. A computer-implemented method according to claim 2 and claim 3, further comprising:deriving one or more image relationship input data items from the image relationship data; andincluding the image relationship input data in the prompt data.

5. A computer-implemented method according to claim 4, comprising receiving user input via a common graphical user interface component to specify relationships between Figures whereby to control both the relationship input data items and at least some of the reuse settings for the part reference values.

6. A computer-implemented method according to any preceding claim, further comprising:generating a candidate part reference value for the recurring part for use if the recurring part is to be indicated as depicted in the second image, wherein the candidate part reference value is at least partially controlled to correspond to a part reference value for the recurring part when depicted in the first image;displaying a preview of the generated candidate part reference value such that the generated candidate part reference value can be selected by user input;receiving user input adding a callout to the second image and associating the generated candidate part reference value with the recurring part when depicted in the second image.

7. A computer-implemented method according to claim 6, further comprising: generating a further candidate part reference value for a so-far undepicted part that has not yet been indicated as depicted in the set of images for use if the so-far undepicted part is to be indicated as depicted in the second image, wherein the candidate part reference value is at least partially controlled in accordance with a predetermined increment from a last-used part reference value in a series of part reference values;displaying a preview of the further generated candidate part reference value such that the further generated candidate part reference value can be selected by user input;receiving user input adding a callout to the second image and associating the further generated candidate part reference value with the so-far undepicted part when indicated as depicted in the second image.

8. A computer-implemented method according to any preceding claim, wherein determining the second subset of parts comprises displaying the second image in an image editor and iteratively:receiving user input adding a callout to the image and associating the added callout with a part in the set of parts; andpopulating the second subset of parts with the part associated with the callout.

9. A computer-implemented method according to claim 6, claim 7 and claim 8, further comprising:displaying part reference values for the second subset of parts to a user; inhibiting amendment of the part reference value for the recurring part; and allowing amendment of the part reference value for the so-far undepicted part.

10. A computer-implemented method according to any preceding claim, wherein the set of images are a set of figures for a patent specification and the part reference value are used in the figures and a specific description of the image for inclusion in the patent specification.

11. A computer-implemented method according to any preceding claim, wherein the reuse setting data comprises one or more element-level reuse control data configured to specify whether at least a modulo remainder portion of a part reference value assigned to a recurring part in a preceding image is to be reused in a later image, wherein the one or more element-level reuse control data include data records for at least one of:a part level, so as to be associated with a respective part in the set of parts;a group-of-parts level, so as to be associated with a respective group of parts in the set of parts;an image level, so as to be associated with a respective image in the set of images; anda group-of-images level, so as to be associated with a respective group of image in the set of images.

12. A computer-implemented method according to claim 11, wherein the part reference value can be expressed as 100 x H + R, where H represents a high-order portion and R is an integer between 0 and 99, R representing the modulo remainder portion.

13. A computer-implemented method according to claim 12, wherein in at least one numbering control configuration, the one or more element-level reuse control data are configured to specify whether both the high-order portion and the modulo remainder portion of a part reference value assigned to a recurring part in a preceding image is to be reused in a later image.

14. A computer-implemented method according to any preceding claim, wherein the reuse setting data comprises one or more numbering mode settings, wherein the one or more numbering mode settings are associated with at least one of:an image in the set of images;a subset of images in the set of images; andthe set of images generally.

15. A computer-implemented method according to claim 14, wherein the one or more numbering mode settings comprise one or more high-order numbering mode settings associated with a high-order numbering mode in which a high-order number portion of a part reference value is controlled in accordance with the high-order numbering mode.

16. A computer-implemented method according to claim 15, wherein the images inthe set of images are associated with respective image indices, and wherein the one or more high-order numbering mode settings comprise at least one high-order numbering mode setting in which a high-order number portion of a part reference value is generated in dependence on the image index associated with the image in which a part reference value is to be used to indicate that a part is depicted in the image.

17. A computer-implemented method according to claim 16, wherein the one or more high-order numbering mode settings comprise:a first-appearance mode setting associated with a first-appearance high-order numbering mode in which the high-order number portion of a part reference value for a recurring part is fixedly controlled in accordance with a lowest image index for an image in which the recurring part appears, for each of a plurality of different images in which the recurring part appears; and / oran image-indexed mode setting associated with an image-indexed high-order numbering mode in which the high-order number portion of a part reference value for a recurring part is dynamically controlled in accordance with the image index for each of a plurality of different images in which the recurring part appears.

18. A computer-implemented method according to any of claims 16 or 17, wherein the one or more high-order numbering mode settings comprise:an incremental mode setting associated with an incremental high-order numbering mode in which the high-order number portion of a part reference value for a recurring part is dynamically controlled in accordance with a predetermined increment in a high-order number series, irrespective of the image index of the images in which the recurring part appears.

19. A computer-implemented method according to claim 18, wherein in the incremental mode, the part reference value remains fixed for the recurring part when it is depicted within a defined subset of images and is incremented in the high-order number portion when the recurring part appears in an image outside the defined subset.

20. A computer-implemented method according to claim 11 and claim 19, in whichthe at least one high-order numbering mode setting is associated with the set of images generally, and wherein the setting data comprises a plurality of element-level reuse control data records each defining a respective subset of images in the set of images.

21. A computer-implemented method according to claim 20, wherein each of the plurality of element-level reuse control data records are associated with individual images and define a respective subset of images in the set of images from which at least a portion of a part reference value is to be reused for a recurring part depicted in the individual image.

22. A computer-implemented method according to any preceding claim, further comprising storing the obtained part reference values in association with a parts list, and automatically updating the part reference values in response to changes in the reuse setting data.

23. A computer-implemented method according to any preceding claim, further comprising performing a part reference value maintenance function which automatically updates part reference values in response to a change in the set of images.

24. A computer-implemented method according to claim 23, comprising performing the part reference value maintenance function in response to deletion of an image, and inhibiting deletion of an image identified as a parent image in an image relationship used to define a part reference value in a child image.

25. A computer-implemented method according to claim 23 or 24, comprising performing the part reference value maintenance function in response to reordering of images in the set of images, and inhibiting reordering of an image identified as a child image in an image relationship used to define a part reference value in the child image if the reordering would reorder the child image in front of the parent image.

26. A computer-implemented method according to any preceding claim, wherein the reuse setting data further comprises a manual override indicator that permits a userto manually specify a part reference value for the recurring part, thereby overriding an automatically-generated part reference value.

27. A computer-implemented method according to any preceding claim, further comprising reordering parts depicted in at least one image, the reordering triggering recalculation of the part reference values for the recurring part in dependence on the reuse setting data.

Images