Preprocessing method and device of job image, equipment and storage medium

Abstract

The invention discloses a job image preprocessing method, device and equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: constructing a preprocessing network architecture of a job image in advance; in the process of performing forward reasoning on the to-be-processed job image by using the feature extraction network, feature maps output by the shared feature layer are respectively butted to different task branch networks so as to obtain prediction information output by the different task branch networks; the different task branch networks comprise a first type of task branch network and a second type of task branch network; and preprocessing the job image according to the prediction information output by the different task branch networks to obtain an image preprocessing result. In the whole process, a serial multi-model architecture is used for preprocessing the job image, the preprocessing process of the job image is simplified, the preprocessing efficiency of the job image and the resource adaptability of terminal equipment can be considered at the same time, and the expected requirement of a job correction scene for image preprocessing is effectively met.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for preprocessing job images. Background Technology

[0002] In the homework grading scenario of intelligent learning devices, the learning machine, as the core terminal device, needs to support users to submit homework via image upload and complete automated grading. Preprocessing of the homework images is a crucial step in ensuring the accuracy of subsequent grading. Its core objective is to extract clear, well-organized, and effective main content of the homework from the original images uploaded by the user, laying the foundation for subsequent core grading processes such as character recognition and question matching.

[0003] The existing preprocessing workflow for assignment images typically relies on a serial multi-model architecture. The specific operation logic is as follows: First, the subject detection model is called to perform object detection on the original uploaded assignment image, identify and crop out the main subject area containing the assignment content, such as exam papers, exercise book pages, etc., and remove irrelevant interfering elements such as background and clutter. Second, the rotation judgment model analyzes the tilt angle of the cropped main subject image and performs rotation correction operation based on the analysis results to ensure that the assignment content is in a horizontal and regular state. Finally, the image validity judgment model is activated to verify whether the image after cropping and rotation processing meets the grading requirements, such as excluding invalid data such as blank images, blurry images, and non-assignment images. Only when the image is determined to be valid will the subsequent assignment grading process be triggered.

[0004] However, the aforementioned preprocessing workflow for job images has the following limitations: Firstly, the serial multi-model architecture not only increases the overall system complexity but also generates collaborative errors due to the coupling and correlation defects between the inference results of various models, affecting the accuracy of subsequent grading results. Secondly, the serial multi-model architecture requires sequentially loading model parameters and distributively executing inference calculations, making its operation highly dependent on the hardware computing resources and system space resources of the terminal device, reducing the preprocessing efficiency of job images and making it difficult to meet the expected requirements of job grading scenarios. In summary, the above two limitations cannot simultaneously achieve both preprocessing efficiency and terminal adaptability for job images, making it difficult to meet the expected requirements of job grading scenarios. Summary of the Invention

[0005] In view of this, this application provides a method and apparatus for preprocessing job images, the main purpose of which is to solve the problem that the process of preprocessing job images using a serial multi-model architecture in the prior art cannot take into account both the preprocessing efficiency and terminal adaptability of job images, and is difficult to meet the expected requirements of job grading scenarios for job images.

[0006] According to a first aspect of this application, a preprocessing method for job images is provided, comprising: A preprocessing network architecture for pre-constructed task images is provided, the preprocessing network architecture including a feature extraction network and multiple task branch networks, the feature extraction network having a shared feature layer, and the feature extraction network being connected to the multiple task branch networks through the shared feature layer; During the forward inference process of the task image to be processed using the feature extraction network, the feature map output by the shared feature layer is connected to different task branch networks to obtain the prediction information output by different task branch networks. The different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, and the second type of task branch network reuses the feature blocks obtained by dividing the feature map into blocks to perform the corresponding task operation. The task image is preprocessed based on the prediction information output by the different task branch networks to obtain the image preprocessing result.

[0007] Furthermore, the step of connecting the feature maps output by the shared feature layer to different task branch networks to obtain the prediction information output by the different task branch networks includes: The feature map output by the shared feature layer is completely input into the first type of task branch network, so that the first type of task branch network performs global feature recognition operation on the task image based on the complete feature map, and obtains the position prediction information of the main region in the task image. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the second type of task branch network so that the second type of task branch network performs local feature recognition operation on the task image based on the multiple feature blocks to obtain the attribute prediction information of each feature block in the task image.

[0008] Further, the second type of task branch network includes a directional task branch network and an effectiveness task branch network. The feature map output by the shared feature layer is divided into multiple feature blocks according to a grid topology. These multiple feature blocks are input into the second type of task branch network, which performs local feature recognition operations on the task image based on these multiple feature blocks to obtain attribute prediction information for each feature block in the task image, including: The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the directional task branch network so that the directional task branch network can identify the tilt angle of local areas in the task image based on the multiple feature blocks. Based on the confidence of each feature block in different directions, the directional prediction information of each feature block in the task image is determined. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the validity task branch network so that the validity of local regions in the task image can be identified by the validity task branch network based on the multiple feature blocks. The validity prediction information of each feature block in the task image is determined according to the validity confidence of each feature block.

[0009] Further, after dividing the feature map output by the shared feature layer into multiple feature blocks according to a grid topology, and inputting the multiple feature blocks into the directional task branch network to identify the tilt angle of local regions in the task image based on the multiple feature blocks, and determining the directional prediction information of each feature block in the task image according to the confidence of each feature block in different directions, the method further includes: Iterate through the image to obtain the set of neighboring feature blocks for each feature block in different directions; Based on the direction prediction information of each neighbor feature block in the neighbor feature block set, the direction prediction information of each feature block in the working image is adjusted.

[0010] Further, adjusting the direction prediction information of each feature block in the job image based on the direction prediction information of each neighbor feature block in the neighbor feature block set includes: If the orientation prediction information of all neighboring feature blocks in the neighboring feature block set is consistent, the orientation prediction information of the corresponding feature block in the job image is adjusted to be consistent with the orientation prediction information of the neighboring feature blocks; If there is inconsistency in the orientation prediction information of neighboring feature blocks in the set of neighboring feature blocks, the orientation prediction information of the corresponding feature blocks in the working image shall remain unchanged.

[0011] Further, after dividing the feature map output by the shared feature layer into multiple feature blocks according to a grid topology, and inputting the multiple feature blocks into the validity task branch network to identify the validity of local regions in the task image based on the multiple feature blocks, and determining the validity prediction information of each feature block in the task image according to the validity confidence of each feature block, the method further includes: Iterate through the image to obtain the set of neighboring feature blocks for each feature block in different directions; Based on the validity prediction information of each neighbor feature block in the neighbor feature block set, the validity prediction information of each feature block in the job image is adjusted.

[0012] Further, adjusting the validity prediction information of each feature block in the job image based on the validity prediction information of each neighbor feature block in the neighbor feature block set includes: If the validity prediction information of all neighbor feature blocks in the neighbor feature block set is consistent, the validity prediction information of the corresponding feature block in the job image is adjusted to be consistent with the validity prediction information of the neighbor feature blocks. If there is inconsistency in the validity prediction information of neighboring feature blocks in the set of neighboring feature blocks, the validity prediction information of the corresponding feature blocks in the job image shall remain unchanged.

[0013] Further, the step of preprocessing the task image based on the prediction information output by the different task branch networks to obtain the image preprocessing result includes: Identify valid feature blocks in the work image; The image is preprocessed based on the attribute prediction information of the effective feature blocks in the image to obtain the image preprocessing result.

[0014] Further, determining valid feature blocks in the work image includes: The predicted location information of the main body region in the work image is matched with the spatial location of each feature block in the work image to filter out the feature blocks whose spatial location is within the main body region as valid feature blocks. Accordingly, the preprocessing of the task image based on the attribute prediction information of the effective feature blocks in the task image to obtain the image preprocessing result includes: Based on the direction prediction information of the effective feature blocks in the task image, a majority vote is performed on the task image to obtain the direction statistics result; Based on the direction statistics, the local area of ​​the work image is subjected to direction correction processing to obtain the image preprocessing result; Accordingly, the preprocessing of the task image based on the attribute prediction information of the effective feature blocks in the task image to obtain the image preprocessing result includes: The majority vote statistics of the job image are performed based on the validity prediction information of the effective feature blocks in the job image to obtain the validity statistics results; Based on the validity statistics, the local area of ​​the job image is subjected to validity screening to obtain the image preprocessing result.

[0015] Furthermore, after performing majority voting statistics on the task image based on the direction prediction information of effective feature blocks in the task image to obtain the direction statistics result, the method further includes: If there are at least two directions with the highest statistical proportion in the direction statistics results, then the direction with the highest total confidence is determined as the direction statistics result based on the confidence statistics results of the effective feature blocks in the at least two directions. Accordingly, after performing majority voting statistics on the job image based on the validity prediction information of valid feature blocks in the job image to obtain the validity statistics result, the method further includes: If the statistical proportions of valid confidence and invalid confidence are the same in the validity statistics, then the validity identifier with the highest total confidence is determined as the validity statistics result based on the statistical results of valid confidence and invalid confidence of valid feature blocks.

[0016] According to a second aspect of this application, a preprocessing apparatus for job images is provided, comprising: A construction unit is used to pre-construct a preprocessing network architecture for a task image. The preprocessing network architecture includes a feature extraction network and multiple task branch networks. The feature extraction network is provided with a shared feature layer, and the feature extraction network is connected to the multiple task branch networks through the shared feature layer. The acquisition unit is used to connect the feature map output by the shared feature layer to different task branch networks during the forward inference process of the task image to be processed using the feature extraction network, so as to obtain the prediction information output by the different task branch networks. The different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, and the second type of task branch network reuses the feature blocks obtained by dividing the feature map into blocks to perform the corresponding task operation. The preprocessing unit is used to preprocess the task image based on the prediction information output by the different task branch networks to obtain the image preprocessing result.

[0017] Furthermore, the acquisition unit includes: The first recognition module is used to input the feature map output by the shared feature layer into the first type of task branch network, so that the first type of task branch network can perform global feature recognition operation on the task image based on the complete feature map to obtain the position prediction information of the main region in the task image. The second recognition module is used to divide the feature map output by the shared feature layer into multiple feature blocks according to the grid topology, and input the multiple feature blocks into the second type of task branch network, so that the second type of task branch network performs local feature recognition operation on the task image based on the multiple feature blocks to obtain the attribute prediction information of each feature block in the task image.

[0018] Furthermore, the second type of task branch network includes a directional task branch network and a validity task branch network, and the second identification module is specifically used for: The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the directional task branch network so that the directional task branch network can identify the tilt angle of local areas in the task image based on the multiple feature blocks. Based on the confidence of each feature block in different directions, the directional prediction information of each feature block in the task image is determined. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the validity task branch network so that the validity of local regions in the task image can be identified by the validity task branch network based on the multiple feature blocks. The validity prediction information of each feature block in the task image is determined according to the validity confidence of each feature block.

[0019] Furthermore, the second identification module is specifically used for: The feature map output by the shared feature layer is divided into multiple feature blocks according to a grid topology. These multiple feature blocks are then input into a directional task branch network. The directional task branch network identifies the tilt angle of local regions in the task image based on these multiple feature blocks. After determining the directional prediction information of each feature block in the task image based on the confidence level of each feature block in different directions, the set of neighboring feature blocks in different directions for each feature block in the task image is obtained through traversal. The directional prediction information of each feature block in the task image is then adjusted based on the directional prediction information of each neighboring feature block in the set of neighboring feature blocks.

[0020] Furthermore, the second identification module is specifically used for: If the orientation prediction information of all neighboring feature blocks in the neighboring feature block set is consistent, the orientation prediction information of the corresponding feature block in the job image is adjusted to be consistent with the orientation prediction information of the neighboring feature blocks; If there is inconsistency in the orientation prediction information of neighboring feature blocks in the set of neighboring feature blocks, the orientation prediction information of the corresponding feature blocks in the working image shall remain unchanged.

[0021] Furthermore, the second identification module is specifically used for: The feature map output by the shared feature layer is divided into multiple feature blocks according to a grid topology. These multiple feature blocks are then input into the validity task branch network. The validity task branch network identifies the validity of local regions in the task image based on these multiple feature blocks. After determining the validity prediction information of each feature block in the task image according to the validity confidence of each feature block, the set of neighboring feature blocks in different directions for each feature block in the task image is obtained through traversal. The validity prediction information of each feature block in the task image is then adjusted according to the validity prediction information of each neighboring feature block in the set of neighboring feature blocks.

[0022] Furthermore, the second identification module is specifically used for: If the validity prediction information of all neighbor feature blocks in the neighbor feature block set is consistent, the validity prediction information of the corresponding feature block in the job image is adjusted to be consistent with the validity prediction information of the neighbor feature blocks. If there is inconsistency in the validity prediction information of neighboring feature blocks in the set of neighboring feature blocks, the validity prediction information of the corresponding feature blocks in the job image shall remain unchanged.

[0023] Furthermore, the preprocessing unit includes: The determination module is used to identify valid feature blocks in the work image; The preprocessing module is used to preprocess the job image based on the attribute prediction information of the effective feature blocks in the job image to obtain the image preprocessing result.

[0024] Furthermore, the determining module is specifically used for: The predicted location information of the main body region in the work image is matched with the spatial location of each feature block in the work image to filter out the feature blocks whose spatial location is within the main body region as valid feature blocks. Accordingly, the preprocessing module is specifically used for: Based on the direction prediction information of the effective feature blocks in the task image, a majority vote is performed on the task image to obtain the direction statistics result; Based on the direction statistics, the local area of ​​the work image is subjected to direction correction processing to obtain the image preprocessing result; Accordingly, the determining module is specifically used for: The majority vote statistics of the job image are performed based on the validity prediction information of the effective feature blocks in the job image to obtain the validity statistics results; Based on the validity statistics, the local area of ​​the job image is subjected to validity screening to obtain the image preprocessing result.

[0025] Furthermore, the determining module is specifically used for: After performing majority voting statistics on the work image based on the direction prediction information of the effective feature blocks in the work image to obtain the direction statistics result, if there are at least two directions with the highest statistical proportion in the direction statistics result, then the direction with the highest total confidence is determined as the direction statistics result based on the confidence statistics results of the effective feature blocks in the at least two directions. Accordingly, the determining module is further configured to: After performing majority voting statistics on the job image based on the validity prediction information of the valid feature blocks in the job image to obtain the validity statistics result, if the statistical proportions of valid confidence and invalid confidence in the validity statistics result are the same, then based on the valid confidence statistics result and the invalid confidence statistics result of the valid feature blocks, the validity identifier with the highest total confidence is determined as the validity statistics result.

[0026] According to a third aspect of this application, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps described in the first aspect above.

[0027] According to a fourth aspect of this application, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0028] By utilizing the above technical solutions, this application provides a preprocessing method, apparatus, and device for job images. Compared with the existing technology that uses a serial multi-model architecture to preprocess job images, this application pre-constructs a preprocessing network architecture for job images. The preprocessing network architecture includes a feature extraction network and multiple task branch networks. The feature extraction network is equipped with a shared feature layer, and the feature extraction network is connected to multiple task branch networks through the shared feature layer. During the forward inference process of the job image to be processed using the feature extraction network, the feature map output by the shared feature layer is respectively connected to different task branch networks to obtain the prediction information output by different task branch networks. The different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, and the second type of task branch network reuses the feature blocks obtained by dividing the feature map into blocks to perform the corresponding task operation. The job image is preprocessed according to the prediction information output by different task branch networks to obtain the image preprocessing result. The entire process replaces the serial multi-model architecture with a parallel preprocessing architecture using a multi-branch shared feature network. This eliminates the need for repeated feature extraction operations on the original job images, allowing for single feature extraction through the shared feature layer to support multiple preprocessing tasks. This architecture not only simplifies the job image preprocessing workflow and significantly reduces the computational load during the inference stage, but also balances preprocessing efficiency with resource adaptability to terminal devices, effectively meeting the expected image preprocessing requirements of job grading scenarios.

[0029] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0030] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a preprocessing method for job images provided in an embodiment of this application; Figure 2 yes Figure 1 A flowchart illustrating a specific implementation method for step 102; Figure 3 This is a schematic diagram of the preprocessing network architecture for job images provided in an embodiment of this application; Figure 4 This is a schematic diagram of the operation image provided in the embodiment of this application; Figure 5This is a schematic diagram of the main area obtained after the task image provided in this application embodiment has been processed by the first type of task branch network; Figure 6 This is a schematic diagram of the main area obtained after the task image provided in this application embodiment has been processed by the second type of task branch network; Figure 7 yes Figure 1 A flowchart illustrating a specific implementation method for step 103; Figure 8 This is a schematic diagram of the structure of a preprocessing device for job images provided in an embodiment of this application; Figure 9 This is a schematic diagram of the device structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0031] The invention will now be discussed with reference to several exemplary embodiments. It should be understood that these embodiments are described merely to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0032] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment". The term "another embodiment" is to be interpreted as "at least one other embodiment".

[0033] In related technologies, the preprocessing process of assignment images typically relies on a serial multi-model architecture. In this architecture, the models are trained independently. However, with the increasing prevalence of intelligent learning machines and the growing demand for real-time and convenient assignment grading, the serial multi-model architecture cannot meet the needs of practical applications. On the one hand, it not only increases the overall complexity of the system but also generates collaborative errors due to the coupling and correlation defects between the inference results of various models, affecting the accuracy of subsequent grading results. On the other hand, the serial multi-model architecture requires sequentially loading model parameters and distributively executing the inherent logic of inference calculations. Its operation is highly dependent on the hardware computing resources and system space resources of the terminal device, reducing the preprocessing efficiency of assignment images and making it difficult to meet the expected requirements of assignment grading scenarios.

[0034] To address this problem, this embodiment provides a preprocessing method for job images, such as... Figure 1 As shown, the method includes the following steps: 101. Pre-construct the preprocessing network architecture for the job images.

[0035] In this embodiment, the preprocessing network architecture includes a feature extraction network and multiple task branch networks. The feature extraction network has a shared feature layer, which is connected to the multiple task branch networks. Typically, the preprocessing network architecture is a pre-designed and trained deep learning model used to perform automatic blurring preprocessing tasks on the job image, such as main region localization, local orientation adjustment, and effective region selection. Specifically, in the preprocessing network architecture, the feature extraction network serves as the base network, primarily used to learn features from the job image to output feature maps representing the essential information of the job image, such as edge features, texture features, and speech features. The multiple task branch networks, based on the feature maps output by the feature extraction network, execute different preprocessing sub-tasks in parallel, such as a task branch for identifying the main region in the job image, a task branch for angle recognition in the job image, and a task branch for effectiveness recognition in the job image.

[0036] As a key output of the feature extraction layer, the shared feature layer is typically selected from the tail layer of the feature extraction network. This is because the tail layer has a wider receptive field, enabling it to capture the global semantic features of the task image. The last layer in the feature extraction network is preferred because its output feature map is shared by all task branches, avoiding redundant computation by each branch network extracting features individually and improving the network's computational efficiency. It's understandable that the feature extraction network does not directly interact with the task branch networks, but rather uses the shared feature layer as a connection layer to ensure that all task branches execute subtasks based on unified underlying features, guaranteeing consistency and relevance in multi-task processing.

[0037] In practical applications, the structure of feature extraction networks can be designed according to the requirements of the scenario. Differences in the design of convolutional kernel sizes will result in different receptive field sizes in the last layer of the network. For example, when using ResNet as a feature extraction network, the receptive field of its last layer can cover the entire image, while when using a lightweight MobileNet, the receptive field of its last layer is relatively small. In this case, a network layer with a suitable receptive field can be selected from the tail layers of the feature extraction network as a shared feature layer to ensure the effectiveness of feature reuse.

[0038] 102. During the forward inference process of the task image to be processed using the feature extraction network, the feature maps output by the shared feature layer are respectively connected to different task branch networks to obtain the prediction information output by different task branch networks.

[0039] In this embodiment, after the task image to be processed is input into the feature extraction network, the forward inference operation process is initiated. Generally, the feature extraction network consists of multiple cascaded network structures such as convolutional layers, pooling layers, and activation layers. During the inference process, the pixel information of the task image is transformed layer by layer. First, the low-level edge and texture detail features of the task image are extracted, and then high-level features that can represent the semantic information of the image are generated through multi-layer operation fusion. When the feature extraction process reaches the preset shared feature layer, the shared feature layer outputs a unified high-dimensional feature map. Here, the shared feature layer is usually the tail network layer in the feature extraction network. Its receptive field size is selected and adapted to simultaneously cover the local details and global features of the task image, meeting the feature requirements of multi-task branches.

[0040] Furthermore, the feature maps output by the shared feature layer are synchronously connected to multiple pre-defined task branch networks in the preprocessing network architecture. Each task branch network executes differentiated prediction tasks in parallel based on a unified feature map, eliminating the need for separate feature extraction of the original job images and effectively improving inference efficiency.

[0041] Considering the differentiated needs of image preprocessing tasks, different task branch networks are categorized into two types. The first type reuses complete feature maps to perform corresponding task operations; these are global-granularity preprocessing tasks, such as locating the main region of an image, which relies on the global semantic information of the image. The second type reuses feature blocks obtained by partitioning the feature map to perform corresponding task operations; these are local-granularity preprocessing tasks, such as orientation correction and validity screening, which require fine-grained judgment of local regions. By classifying the task branch networks, different task branch networks can be matched with feature reuse methods that align with their granularity requirements, ensuring the effectiveness of preprocessing tasks from the outset.

[0042] For example, the feature map output by the shared feature layer is divided into Each feature block is processed by the task budget of the second-class task branch network, and the corresponding output is... The attribute prediction information of each feature block. In practical applications, the grid division can be adjusted according to the size ratio between the original image and the main subject area.

[0043] In practical applications, the multi-task branch network in the preprocessing network architecture has clear modular characteristics. If a new preprocessing task is added later, only the type of the new preprocessing task needs to be determined, and the corresponding feature usage method can be reused without reconstructing the entire feature extraction process. For example, if the new preprocessing task is a global task classified by job type, it can be classified into the first type of task branch network, and the execution logic of the first type of task branch network can be reused accordingly. If the new preprocessing task is local region blur detection, it can be classified into the second type of branch network, and the execution logic of the second type of task branch network can be reused accordingly.

[0044] 103. Based on the prediction information output by the different task branch networks, the task image is preprocessed to obtain the image preprocessing result.

[0045] In this embodiment, the prediction information mainly includes global prediction information output by the first type of task branch network and local prediction information output by the second type of task branch network. The global prediction information can be the prediction information of the main body region in the task image, including but not limited to spatial location data such as the boundary coordinates and area range of the main body region, which is used to define the core range of preprocessing. The local prediction information can be the attribute prediction information of each feature block, including but not limited to tilt direction and confidence, effective identification and confidence, etc., which is used to guide the fine processing of local regions.

[0046] Specifically, based on the location prediction information of the main area, invalid areas at the edges of the work image, such as blank page margins and binding marks, can be cropped to focus on the core work area. Based on the attribute prediction information of each feature library, the attributes of local areas of the work image can be adjusted. For example, local areas can be rotated and corrected so that the layout direction of the answer area is unified to the preset standard direction. Invalid interference areas in local areas can also be removed so that only the valid answer area is retained in the work image.

[0047] The preprocessing method for job images provided in this application differs from existing technologies that use a serial multi-model architecture for preprocessing job images. This application pre-constructs a preprocessing network architecture for the job image, including a feature extraction network and multiple task branch networks. The feature extraction network incorporates a shared feature layer and is connected to the multiple task branch networks through this shared feature layer. During the forward inference process of the job image to be processed using the feature extraction network, the feature maps output by the shared feature layer are respectively connected to different task branch networks to obtain prediction information output by these networks. These different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform corresponding task operations, while the second type of task branch network reuses feature blocks obtained by partitioning the feature map to perform corresponding task operations. The job image is preprocessed based on the prediction information output by the different task branch networks to obtain the image preprocessing result. By replacing the serial multi-model architecture with a parallel preprocessing architecture using a multi-branch shared feature network, the entire process eliminates the need for repeated feature extraction operations on the original job image. Multiple preprocessing task requirements can be supported by a single feature extraction operation using the shared feature layer. This architecture not only simplifies the preprocessing process of job images and significantly reduces the computational load in the inference stage, but also balances the preprocessing efficiency of job images with the resource adaptability of terminal devices, effectively meeting the expected requirements of image preprocessing in job grading scenarios.

[0048] In practical applications, the granularity requirements of different preprocessing tasks differ fundamentally. To accurately match the characteristics of the preprocessing task, specifically... Figure 2 As shown, step 102 above includes the following steps: 102A. The feature map output by the shared feature layer is completely input into the first type of task branch network, so that the first type of task branch network performs global feature recognition operation on the task image based on the complete feature map to obtain the position prediction information of the main region in the task image; 102B. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the second type of task branch network so that the second type of task branch network performs local feature recognition operation on the task image based on the multiple feature blocks to obtain the attribute prediction information of each feature block in the task image.

[0049] The first type of task branch network corresponds to global granularity tasks. These tasks require capturing the global semantic associations and layout features of the image, such as the distribution of answer areas in a full-page assignment image and the boundary division between the main content and the edge blank space. The complete feature map preserves the global continuity of the assignment image features, enabling the branch network to accurately identify global attributes spanning multiple local regions. If the feature map is divided into grids, it will sever the global semantic information, leading to problems such as boundary offset and missed detection in the localization of the main body region.

[0050] The second type of task branch network corresponds to local granularity tasks. These tasks require fine-grained judgment of a single local region. Each feature block of the grid corresponds to a local sub-region of the task image. The operation based on the feature block can realize the independent analysis of local attributes. If the complete feature map is used directly, the features of adjacent regions will interfere with each other, and it is impossible to accurately distinguish the tilt direction or effective state of each local region.

[0051] Understandably, both types of branches are based on the feature maps output by the same shared feature layer, but they use different feature usage methods to ensure the consistency of the feature space between the global task and the local task. Subsequently, the location prediction information of the main region output by the first type of task branch network can be directly used to filter the grid feature blocks output by the second type of task branch network, achieving seamless collaboration of multi-task results and avoiding contradictions in results caused by different feature sources.

[0052] As one possible implementation, the second type of task branch network mentioned above includes a directional task branch network and an effectiveness task branch network. Specifically, step 102B includes the following steps: The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the directional task branch network so that the directional task branch network can identify the tilt angle of local areas in the task image based on the multiple feature blocks. According to the confidence of each feature block in different directions, the directional prediction information of each feature block in the task image is determined.

[0053] The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the validity task branch network so that the validity of local regions in the task image can be identified by the validity task branch network based on the multiple feature blocks. The validity prediction information of each feature block in the task image is determined according to the validity confidence of each feature block.

[0054] To ensure the regularity of the blocks, the grid topology structure can use the set grid size as the division standard to divide the complete high-dimensional feature map output by the shared feature layer into multiple non-overlapping local feature blocks of uniform size. A one-to-one mapping relationship is established between each feature block and a local sub-region in the work image. Each feature block uniquely corresponds to a local sub-region in the work image, providing a standardized analysis unit for subsequent local region tilt angle recognition and validity determination.

[0055] Furthermore, all the segmented feature blocks are batch-input into the orientation task branch network and the validity task branch network. The orientation task branch network is designed for the directional characteristics of local features, enabling it to independently analyze key information such as texture direction and edge orientation of each feature block. This allows it to identify the tilt angle of the local region and obtain the confidence level of each feature block in different directions, including common tilt angles such as 0°, ±15°, and ±30°. Higher confidence levels indicate a greater probability that the feature block belongs to the corresponding direction. The direction with the highest confidence level is ultimately selected as the directional prediction information for the local region corresponding to the feature block. The validity task branch network is designed for the semantic characteristics of local features, enabling it to independently analyze the content features of each feature block and determine whether its corresponding local sub-region of the assignment image contains valid information. Valid information needs to be defined in conjunction with the assignment image scene, such as areas containing handwritten notes or answers. Invalid information typically refers to blank areas, stains, irrelevant graffiti, and other distracting content. Similarly, the validity task branch network outputs the validity confidence of each feature block. The higher the validity confidence, the greater the probability that the region corresponding to the feature block is a valid region. Finally, combined with the preset confidence threshold, the validity prediction information of each feature block can be determined. Specifically, if the validity confidence of the feature block is greater than or equal to the threshold, the attribute label corresponding to the feature block is determined to be valid; otherwise, the attribute label corresponding to the feature block is determined to be invalid.

[0056] In practical application scenarios, the network architecture for preprocessing job images is shown in reference 3. The corresponding job image preprocessing process includes the following steps: First, the job image to be processed is input into the feature extraction network. During the forward inference process of the feature extraction network, the shared feature layer outputs a feature map, which is then input into the first type of task branch network and the second type of task branch network in parallel.

[0057] The first type of task branch network directly takes the complete feature map as input and performs global prediction based on the complete feature map to obtain the main output head, which is the position prediction information of the main region in the task image. Before the feature map is input to the second type of task branch network, the feature map is divided into multiple feature blocks according to the grid topology. Then, the second type of task branch network takes the multiple feature blocks as input and performs local prediction based on the multiple feature blocks to obtain the local output head.

[0058] The second type of task branch network includes a direction task branch network and an effectiveness task branch network. Accordingly, the direction task branch network performs local direction prediction based on multiple feature blocks to obtain the direction output head, which is the direction prediction information of the local area in the task image. The effectiveness task branch network performs local effectiveness prediction based on multiple feature blocks to obtain the effectiveness output head, which is the effectiveness prediction information of the local area in the task image.

[0059] The above images of the task to be processed can be used as a reference. Figure 4 As shown, in Figure 4 The original assignment image is a children's picture book, with the main area being the picture book area. Because the picture book is placed on a white plane, the original assignment image is tilted and contains non-core areas, such as the desktop and hands, and is therefore an assignment image that needs to be preprocessed.

[0060] The main body region obtained after processing the above task image through the first type of task branch network can be used as a reference. Figure 5 As shown, in Figure 5 The main body region is a complete rectangular area in the assignment image that contains the core content of the picture book. It covers the cartoon illustrations and all text information of the picture book, removes a small amount of blank areas at the edges of the assignment image, and outputs the boundary coordinates of the main body region accordingly, clarifying its position range in the assignment image, and providing a basis for subsequent spatial matching of effective feature blocks and image cropping.

[0061] The main body region obtained after processing the above task image through the second type of task branch network can be used as a reference. Figure 6 As shown, in Figure 6 The main subject region undergoes local attribute prediction for its internal feature blocks. Specifically, the main subject region is divided into multiple feature blocks according to a grid topology, and all feature blocks within the main subject region are identified as valid feature blocks. Preprocessing of the main subject region is then performed based on the attribute prediction information of these valid feature blocks. It should be noted that the processing scope of the second type of task branch is entirely limited to the main subject region output by the first type of branch, and does not involve invalid areas such as the desktop or hands in the original image, ensuring that processing resources are focused only on valuable content.

[0062] As a feasible application scenario, the orientation prediction of a single feature block relies solely on its own local features, which may lead to low confidence or misjudgment due to image noise or missing local content. However, the tilt direction of the local regions corresponding to adjacent feature blocks in the work image is strongly correlated, and the orientation of neighboring feature blocks serves as a reliable reference, correcting erroneous predictions of isolated feature blocks. Accordingly, after step 201, the above method further includes the following steps: traversing and obtaining the set of neighboring feature blocks in different directions for each feature block in the work image; adjusting the orientation prediction information of each feature block in the work image based on the orientation prediction information of each neighboring feature block in the neighboring feature block set. Specifically, if the orientation prediction information of all neighboring feature blocks in the neighboring feature block set is consistent, the orientation prediction information of the corresponding feature block in the work image is adjusted to be consistent with the orientation prediction information of the neighboring feature blocks; if the orientation prediction information of neighboring feature blocks in the neighboring feature block set is inconsistent, the orientation prediction information of the corresponding feature block in the work image remains unchanged.

[0063] When all neighboring feature blocks have the same orientation, it indicates that the true tilt trend of that region in the image is highly reliable. In this case, the central feature block should maintain the same orientation as its neighbors, requiring correction of the orientation prediction information of the corresponding feature blocks in the image to align with that of its neighbors. When all neighboring feature blocks have inconsistent orientations, it indicates that there is no unified reference for the tilt trend of that region in the image. Forcibly adjusting it may introduce new errors. In this case, maintaining the original orientation of the feature blocks is crucial to avoid interference from the fragmented orientations of neighboring feature blocks, which could make the adjusted orientation prediction information even less reliable.

[0064] As a feasible application scenario, the validity prediction of a single feature block relies solely on its own local features, making it susceptible to noise interference or missing content in the work image, resulting in valid regions being misjudged as invalid or invalid regions being misjudged as valid. However, the local regions corresponding to adjacent feature blocks in the work image have strong semantic correlations. By coordinating adjustments with neighboring feature blocks, the validity results of feature blocks within the same semantic region can be kept consistent, avoiding fragmented judgments. Accordingly, after step 202, the above method further includes the following steps: traversing and obtaining the set of neighboring feature blocks in different directions for each feature block in the work image; adjusting the validity prediction information of each feature block in the work image based on the validity prediction information of each neighboring feature block in the neighboring feature block set. Specifically, if the validity prediction information of all neighboring feature blocks in the neighboring feature block set is consistent, the validity prediction information of the corresponding feature block in the work image is adjusted to be consistent with the validity prediction information of the neighboring feature blocks; if the validity prediction information of neighboring feature blocks in the neighboring feature block set is inconsistent, the validity prediction information of the corresponding feature block in the work image remains unchanged.

[0065] When the validity results of all neighboring feature blocks are completely consistent, it indicates that the validity judgment of the region has a high degree of reliability. In this case, the central feature block should maintain consistency with the validity of its neighboring feature blocks. The validity prediction information of the corresponding feature blocks in the work image needs to be corrected and adjusted to be consistent with the validity prediction information of its neighboring feature blocks. When the validity results of all neighboring feature blocks are inconsistent, it indicates that the validity boundary of the region in the work image is blurred and lacks a unified reference. Forcibly adjusting it may introduce new misjudgments. In this case, keeping the validity results of the feature blocks unchanged can avoid interference with the judgment accuracy due to contradictory information from neighboring feature blocks.

[0066] The aforementioned neighbor feature blocks can be defined by setting a fixed-size neighborhood window based on the regularity of the grid topology. The neighborhood window obtains the set of neighbor feature blocks for each feature block in different directions. Typically, the neighborhood window is centered on the feature block, selecting one feature block in each of the top, bottom, left, and right directions, for a total of four neighbor feature blocks. Alternatively, it can be centered on the feature block, selecting one feature block in each of the top, bottom, left, right, and diagonal directions, for a total of eight neighbor feature blocks. If a feature block is located at the image edge, the neighborhood window automatically shrinks, selecting only the neighbor feature blocks containing that feature block. However, if the number of neighbor feature blocks is less than the set number, there is no need to adjust the prediction information for that feature block.

[0067] Understandably, raw work images often contain a large number of worthless interference regions, such as margins, binding marks, stains, and irrelevant graffiti. The feature blocks corresponding to these regions are all invalid feature blocks. Without filtering for valid feature blocks, the interference from these invalid regions will lead to deviations in the fitting of preprocessing parameters. Specifically, for example... Figure 7 As shown, step 103 above includes the following steps: 201. Identify valid feature blocks in the work image.

[0068] 202. Based on the attribute prediction information of the effective feature blocks in the operation image, the operation image is preprocessed to obtain the image preprocessing result.

[0069] In this embodiment, the predicted location information of the main body region output by the first type of task branch network is typically represented by the boundary coordinates of the core region in the task image, representing the value area range of the task image. Similarly, each feature block has corresponding two-dimensional coordinates and the pixel range it covers, which are mapped to local sub-regions of the task image. Specifically, the predicted location information of the main body region in the task image can be spatially matched with each feature block in the task image to filter out feature blocks whose spatial location is within the main body region as valid feature blocks. This process uses the main body region as a spatial filtering box, retaining only feature blocks that fall completely or partially within the main body region, and directly filtering out feature blocks corresponding to invalid regions outside the main body region.

[0070] For example, the boundary of the main region is denoted as region R(x1, y1, x2, y2), where x1 and y1 are the coordinates of the upper left corner, and x2 and y2 are the coordinates of the lower right corner. Traverse all feature blocks and record the center coordinates (xc, yc) of each feature block. If the center coordinates of a feature block satisfy x1≤xc≤x2 and y1≤yc≤y2, then the feature block is determined to be a valid feature block.

[0071] Accordingly, during the preprocessing of the task image based on the attribute prediction information of the effective feature blocks in the task image, a majority vote can be performed on the task image based on the direction prediction information of the effective feature blocks to obtain the direction statistics result; then, direction correction processing is performed on the local area of ​​the task image based on the direction statistics result to obtain the image preprocessing result. It can be understood that each effective feature block has candidate directions and corresponding confidence levels before the voting statistics. Specifically, each effective feature block can be traversed, and the direction with the highest confidence level can be selected as the voting direction for that effective feature block. Then, the voting directions of all effective feature blocks are summarized and counted to obtain the total number of votes for each candidate direction, forming a voting ranking. Finally, the direction with the highest voting ranking is determined as the unified tilt direction of the main area.

[0072] It should be noted that there may be special cases where the voting ranking is tied. Accordingly, after performing majority voting statistics on the work image based on the direction prediction information of the effective feature blocks in the work image, if there are at least two directions with the highest statistical proportion in the direction statistics results, then the direction with the highest total confidence is determined as the direction statistics result based on the confidence statistics results of the effective feature blocks in at least two directions.

[0073] Accordingly, during the preprocessing of the work image based on the attribute prediction information of the effective feature blocks in the work image, a majority vote can be performed on the work image based on the validity prediction information of the effective feature blocks in the work image to obtain the validity statistics result; based on the validity statistics result, a validity screening process is performed on the local areas of the work image to obtain the image preprocessing result. It can be understood that each effective feature block has a validity attribute and a corresponding confidence level before the voting statistics. Specifically, each effective feature block can be iterated through, and the values ​​of the confidence levels of the effective and invalid identifiers can be compared. If the confidence level of the effective identifier is higher, then the effective feature block is voted for as a valid identifier; if the confidence level of the invalid identifier is higher, then it is voted for as an invalid identifier. Then, the total number of votes for the effective and invalid identifiers is counted, and the identifier with the higher number of votes is determined as the validity identifier of the main region.

[0074] It should be noted that there may be special cases where the total number of votes for valid and invalid identifiers is the same. Accordingly, after performing majority voting statistics on the job image based on the validity prediction information of the valid feature blocks in the job image, if the statistical proportions of valid confidence and invalid confidence in the validity statistics are the same, then the valid identifier with the highest total confidence is determined as the validity statistics result based on the statistical results of valid confidence and invalid confidence of the valid feature blocks.

[0075] Furthermore, as a specific implementation of the above method, embodiments of this application provide a preprocessing apparatus for job images, such as... Figure 8 As shown, the device includes: a construction unit 31, an acquisition unit 32, and a preprocessing unit 33.

[0076] Construction unit 31 is used to pre-construct a preprocessing network architecture for the job image. The preprocessing network architecture includes a feature extraction network and multiple task branch networks. The feature extraction network is provided with a shared feature layer, and the feature extraction network is connected to the multiple task branch networks through the shared feature layer. The acquisition unit 32 is used to connect the feature map output by the shared feature layer to different task branch networks during the forward inference process of the task image to be processed using the feature extraction network, so as to obtain the prediction information output by the different task branch networks. The different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, and the second type of task branch network reuses the feature blocks obtained by dividing the feature map into blocks to perform the corresponding task operation. The preprocessing unit 33 is used to preprocess the task image based on the prediction information output by the different task branch networks to obtain the image preprocessing result.

[0077] The preprocessing apparatus for job images provided in this invention, compared with the existing technology that uses a serial multi-model architecture for preprocessing job images, pre-constructs a preprocessing network architecture for job images. This preprocessing network architecture includes a feature extraction network and multiple task branch networks. The feature extraction network has a shared feature layer, which is connected to the multiple task branch networks. During the forward inference process of the job image to be processed using the feature extraction network, the feature map output by the shared feature layer is respectively connected to different task branch networks to obtain prediction information output by different task branch networks. These different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, while the second type of task branch network reuses feature blocks obtained by partitioning the feature map to perform the corresponding task operation. The job image is preprocessed based on the prediction information output by the different task branch networks to obtain the image preprocessing result. The entire process replaces the serial multi-model architecture with a parallel preprocessing architecture of a multi-branch shared feature network, eliminating the need for repeated feature extraction operations on the original job image. Multiple preprocessing task requirements can be supported by a single feature extraction using the shared feature layer. This architecture not only simplifies the preprocessing process of job images and significantly reduces the computational load in the inference stage, but also balances the preprocessing efficiency of job images with the resource adaptability of terminal devices, effectively meeting the expected requirements of image preprocessing in job grading scenarios.

[0078] In practical applications, the acquisition unit includes: The first recognition module is used to input the feature map output by the shared feature layer into the first type of task branch network, so that the first type of task branch network can perform global feature recognition operation on the task image based on the complete feature map to obtain the position prediction information of the main region in the task image. The second recognition module is used to divide the feature map output by the shared feature layer into multiple feature blocks according to the grid topology, and input the multiple feature blocks into the second type of task branch network, so that the second type of task branch network performs local feature recognition operation on the task image based on the multiple feature blocks to obtain the attribute prediction information of each feature block in the task image.

[0079] In practical applications, the second type of task branch network includes a directional task branch network and a validity task branch network. The second recognition module is specifically used for: The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the directional task branch network so that the directional task branch network can identify the tilt angle of local areas in the task image based on the multiple feature blocks. Based on the confidence of each feature block in different directions, the directional prediction information of each feature block in the task image is determined. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the validity task branch network so that the validity of local regions in the task image can be identified by the validity task branch network based on the multiple feature blocks. The validity prediction information of each feature block in the task image is determined according to the validity confidence of each feature block.

[0080] In practical applications, the second recognition module is further used for: The feature map output by the shared feature layer is divided into multiple feature blocks according to a grid topology. These multiple feature blocks are then input into a directional task branch network. The directional task branch network identifies the tilt angle of local regions in the task image based on these multiple feature blocks. After determining the directional prediction information of each feature block in the task image based on the confidence level of each feature block in different directions, the set of neighboring feature blocks in different directions for each feature block in the task image is obtained through traversal. The directional prediction information of each feature block in the task image is then adjusted based on the directional prediction information of each neighboring feature block in the set of neighboring feature blocks.

[0081] In practical applications, the second recognition module is further used for: If the orientation prediction information of all neighboring feature blocks in the neighboring feature block set is consistent, the orientation prediction information of the corresponding feature block in the job image is adjusted to be consistent with the orientation prediction information of the neighboring feature blocks; If there is inconsistency in the orientation prediction information of neighboring feature blocks in the set of neighboring feature blocks, the orientation prediction information of the corresponding feature blocks in the working image shall remain unchanged.

[0082] In practical applications, the second recognition module is further used for: The feature map output by the shared feature layer is divided into multiple feature blocks according to a grid topology. These multiple feature blocks are then input into the validity task branch network. The validity task branch network identifies the validity of local regions in the task image based on these multiple feature blocks. After determining the validity prediction information of each feature block in the task image according to the validity confidence of each feature block, the set of neighboring feature blocks in different directions for each feature block in the task image is obtained through traversal. The validity prediction information of each feature block in the task image is then adjusted according to the validity prediction information of each neighboring feature block in the set of neighboring feature blocks.

[0083] In practical applications, the second recognition module is further used for: If the validity prediction information of all neighbor feature blocks in the neighbor feature block set is consistent, the validity prediction information of the corresponding feature block in the job image is adjusted to be consistent with the validity prediction information of the neighbor feature blocks. If there is inconsistency in the validity prediction information of neighboring feature blocks in the set of neighboring feature blocks, the validity prediction information of the corresponding feature blocks in the job image shall remain unchanged.

[0084] In practical applications, the preprocessing unit includes: The determination module is used to identify valid feature blocks in the work image; The preprocessing module is used to preprocess the job image based on the attribute prediction information of the effective feature blocks in the job image to obtain the image preprocessing result.

[0085] In practical applications, the determining module is specifically used for: The predicted location information of the main body region in the work image is matched with the spatial location of each feature block in the work image to filter out the feature blocks whose spatial location is within the main body region as valid feature blocks. Accordingly, the preprocessing module is specifically used for: Based on the direction prediction information of the effective feature blocks in the task image, a majority vote is performed on the task image to obtain the direction statistics result; Based on the direction statistics, the local area of ​​the work image is subjected to direction correction processing to obtain the image preprocessing result; Accordingly, the determining module is specifically used for: The majority vote statistics of the job image are performed based on the validity prediction information of the effective feature blocks in the job image to obtain the validity statistics results; Based on the validity statistics, the local area of ​​the job image is subjected to validity screening to obtain the image preprocessing result.

[0086] In practical applications, the determining module is further used for: After performing majority voting statistics on the work image based on the direction prediction information of the effective feature blocks in the work image to obtain the direction statistics result, if there are at least two directions with the highest statistical proportion in the direction statistics result, then the direction with the highest total confidence is determined as the direction statistics result based on the confidence statistics results of the effective feature blocks in the at least two directions. Accordingly, the determining module is further configured to: After performing majority voting statistics on the job image based on the validity prediction information of the valid feature blocks in the job image to obtain the validity statistics result, if the statistical proportions of valid confidence and invalid confidence in the validity statistics result are the same, then based on the valid confidence statistics result and the invalid confidence statistics result of the valid feature blocks, the validity identifier with the highest total confidence is determined as the validity statistics result.

[0087] It should be noted that other corresponding descriptions of the functional units involved in the preprocessing apparatus for job images provided in this embodiment can be found in [reference needed]. Figures 1-7 The corresponding descriptions in [the document] will not be repeated here.

[0088] Based on the above, Figures 1-7 Accordingly, this application embodiment also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described method. Figures 1-7 The preprocessing method for the job image shown.

[0089] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0090] Based on the above, Figures 1-7 The method shown, and Figure 8 To achieve the above objectives, the present application also provides a physical device for preprocessing job images, as illustrated in the virtual device embodiment. Specifically, this physical device can be a computer, smartphone, tablet, smartwatch, server, or network device, etc. The physical device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to implement the above-described... Figures 1-7 The preprocessing method for the job image shown.

[0091] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0092] In an exemplary embodiment, see Figure 9 The aforementioned physical device includes a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device. The various functional units can communicate with each other via the bus. The memory stores computer programs, and the processor executes the programs stored in the memory to perform the image preprocessing method described in the above embodiments.

[0093] Those skilled in the art will understand that the physical device structure for preprocessing job images provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0094] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device for preprocessing the aforementioned image processing operations, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0095] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented in hardware. By applying the technical solution of this application, compared with the existing methods, this application replaces the serial multi-model architecture with a parallel preprocessing architecture of a multi-branch shared feature network. This eliminates the need for repeated feature extraction operations on the original job image; a single feature extraction through the shared feature layer is sufficient to support the needs of multiple preprocessing tasks. This architecture not only simplifies the preprocessing process of job images and significantly reduces the computational load in the inference stage, but also simultaneously considers the preprocessing efficiency of job images and the resource adaptability of terminal devices, effectively meeting the expected requirements of image preprocessing in job grading scenarios.

[0096] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0097] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A preprocessing method for job images, characterized in that, include: A preprocessing network architecture for pre-constructed task images is provided, the preprocessing network architecture including a feature extraction network and multiple task branch networks, the feature extraction network having a shared feature layer, and the feature extraction network being connected to the multiple task branch networks through the shared feature layer; During the forward inference process of the task image to be processed using the feature extraction network, the feature map output by the shared feature layer is connected to different task branch networks to obtain the prediction information output by different task branch networks. The different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, and the second type of task branch network reuses the feature blocks obtained by dividing the feature map into blocks to perform the corresponding task operation. The task image is preprocessed based on the prediction information output by the different task branch networks to obtain the image preprocessing result.

2. The method according to claim 1, characterized in that, The step of connecting the feature maps output by the shared feature layer to different task branch networks to obtain the prediction information output by the different task branch networks includes: The feature map output by the shared feature layer is completely input into the first type of task branch network, so that the first type of task branch network performs global feature recognition operation on the task image based on the complete feature map, and obtains the position prediction information of the main region in the task image. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the second type of task branch network so that the second type of task branch network performs local feature recognition operation on the task image based on the multiple feature blocks to obtain the attribute prediction information of each feature block in the task image.

3. The method according to claim 2, characterized in that, The second type of task branch network includes a directional task branch network and an effectiveness task branch network. The feature map output from the shared feature layer is divided into multiple feature blocks according to a grid topology. These multiple feature blocks are input into the second type of task branch network, which performs local feature recognition operations on the task image based on these multiple feature blocks to obtain attribute prediction information for each feature block in the task image, including: The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the directional task branch network so that the directional task branch network can identify the tilt angle of local areas in the task image based on the multiple feature blocks. Based on the confidence of each feature block in different directions, the directional prediction information of each feature block in the task image is determined. The feature map output by the shared feature layer is divided into multiple feature blocks according to the grid topology. The multiple feature blocks are input into the validity task branch network so that the validity of local regions in the task image can be identified by the validity task branch network based on the multiple feature blocks. The validity prediction information of each feature block in the task image is determined according to the validity confidence of each feature block.

4. The method according to claim 3, characterized in that, After dividing the feature map output by the shared feature layer into multiple feature blocks according to a grid topology, and inputting the multiple feature blocks into a directional task branch network to identify the tilt angle of local regions in the task image based on the multiple feature blocks, and determining the directional prediction information of each feature block in the task image according to the confidence level of each feature block in different directions, the method further includes: Iterate through the image to obtain the set of neighboring feature blocks for each feature block in different directions; Based on the direction prediction information of each neighbor feature block in the neighbor feature block set, the direction prediction information of each feature block in the working image is adjusted.

5. The method according to claim 4, characterized in that, The step of adjusting the orientation prediction information of each feature block in the task image based on the orientation prediction information of each neighboring feature block in the neighboring feature block set includes: If the orientation prediction information of all neighboring feature blocks in the neighboring feature block set is consistent, the orientation prediction information of the corresponding feature block in the job image is adjusted to be consistent with the orientation prediction information of the neighboring feature blocks; If there is inconsistency in the orientation prediction information of neighboring feature blocks in the set of neighboring feature blocks, the orientation prediction information of the corresponding feature blocks in the working image shall remain unchanged.

6. The method according to claim 3, characterized in that, After dividing the feature map output by the shared feature layer into multiple feature blocks according to a grid topology, and inputting the multiple feature blocks into the validity task branch network to identify the validity of local regions in the task image based on the multiple feature blocks, and determining the validity prediction information of each feature block in the task image according to the validity confidence of each feature block, the method further includes: Iterate through the image to obtain the set of neighboring feature blocks for each feature block in different directions; Based on the validity prediction information of each neighbor feature block in the neighbor feature block set, the validity prediction information of each feature block in the job image is adjusted.

7. The method according to claim 6, characterized in that, The step of adjusting the validity prediction information of each feature block in the job image based on the validity prediction information of each neighbor feature block in the neighbor feature block set includes: If the validity prediction information of all neighbor feature blocks in the neighbor feature block set is consistent, the validity prediction information of the corresponding feature block in the job image is adjusted to be consistent with the validity prediction information of the neighbor feature blocks. If there is inconsistency in the validity prediction information of neighboring feature blocks in the set of neighboring feature blocks, the validity prediction information of the corresponding feature blocks in the job image shall remain unchanged.

8. The method according to any one of claims 1-7, characterized in that, The step of preprocessing the task image based on the prediction information output by the different task branch networks to obtain the image preprocessing result includes: Identify valid feature blocks in the work image; The image is preprocessed based on the attribute prediction information of the effective feature blocks in the image to obtain the image preprocessing result.

9. The method according to claim 8, characterized in that, The step of determining valid feature blocks in the work image includes: The predicted location information of the main body region in the work image is matched with the spatial location of each feature block in the work image to filter out the feature blocks whose spatial location is within the main body region as valid feature blocks. Accordingly, the preprocessing of the task image based on the attribute prediction information of the effective feature blocks in the task image to obtain the image preprocessing result includes: Based on the direction prediction information of the effective feature blocks in the task image, a majority vote is performed on the task image to obtain the direction statistics result; Based on the direction statistics, the local area of ​​the work image is subjected to direction correction processing to obtain the image preprocessing result; Accordingly, the preprocessing of the task image based on the attribute prediction information of the effective feature blocks in the task image to obtain the image preprocessing result includes: The majority vote statistics of the job image are performed based on the validity prediction information of the effective feature blocks in the job image to obtain the validity statistics results; Based on the validity statistics, the local area of ​​the job image is subjected to validity screening to obtain the image preprocessing result.

10. The method according to claim 9, characterized in that, After performing majority voting statistics on the job image based on the orientation prediction information of effective feature blocks in the job image to obtain the orientation statistics result, the method further includes: If there are at least two directions with the highest statistical proportion in the direction statistics results, then the direction with the highest total confidence is determined as the direction statistics result based on the confidence statistics results of the effective feature blocks in the at least two directions. Accordingly, after performing majority voting statistics on the job image based on the validity prediction information of valid feature blocks in the job image to obtain the validity statistics result, the method further includes: If the statistical proportions of valid confidence and invalid confidence are the same in the validity statistics, then the validity identifier with the highest total confidence is determined as the validity statistics result based on the statistical results of valid confidence and invalid confidence of valid feature blocks.

11. A preprocessing apparatus for job images, characterized in that, include: A construction unit is used to pre-construct a preprocessing network architecture for a task image. The preprocessing network architecture includes a feature extraction network and multiple task branch networks. The feature extraction network is provided with a shared feature layer, and the feature extraction network is connected to the multiple task branch networks through the shared feature layer. The acquisition unit is used to connect the feature map output by the shared feature layer to different task branch networks during the forward inference process of the task image to be processed using the feature extraction network, so as to obtain the prediction information output by the different task branch networks. The different task branch networks include a first type of task branch network and a second type of task branch network. The first type of task branch network reuses the complete feature map to perform the corresponding task operation, and the second type of task branch network reuses the feature blocks obtained by dividing the feature map into blocks to perform the corresponding task operation. The preprocessing unit is used to preprocess the task image based on the prediction information output by the different task branch networks to obtain the image preprocessing result.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the preprocessing method for the job image according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the preprocessing method for the job image according to any one of claims 1 to 10.

Images